Interaction control method, electronic device, storage medium and computer program product

Abstract

The invention provides an interaction control method, electronic equipment, a storage medium and a computer program product, and relates to the technical field of interaction control, and the method comprises the following steps: obtaining first data of each object in interaction objects; determining a first feature of each object and a second feature between the interaction objects based on the first data of the interaction objects; wherein the first feature is used for representing one or more of shape information, attitude information, position information and category information; the second feature is used for representing space action change information between the interaction objects; sending second data obtained by compressing the first feature and the second feature to a second end, so that the second end obtains the first feature, the second feature and a third feature based on the second data, and determines offset information of the target object based on the first feature, the second feature and the third feature; wherein the third feature is used for representing interaction detail information between the interaction objects.

Description

Technical Field

[0001] This application relates to the field of interactive control technology, and in particular to an interactive control method, electronic device, storage medium, and computer program product. Background Technology

[0002] Interactive control technology is widely used in scenarios such as remote operation, virtual reality, and human-computer collaboration. Its core objective is to achieve efficient perception and transmission of the interactive states between multiple objects. To improve the efficiency and accuracy of information transmission during the interaction process, existing technologies often combine multiple methods such as 3D modeling, pose estimation, and feature encoding to describe and compress interactive behavior.

[0003] There are two main problems with related technologies in human-object interaction (HOI) modeling: First, traditional methods based on skeletons or key points cannot fully depict the geometric and semantic relationships between interactive objects, resulting in insufficient interaction perception accuracy. Second, although the use of high-dimensional data such as 3D point clouds or meshes improves modeling capabilities, the large amount of data and high redundancy result in excessively high transmission bitrates, which limits practical applications in low-bandwidth environments and also leads to insufficient interaction perception accuracy. Summary of the Invention

[0004] This application provides an interactive control method, an electronic device, a storage medium, and a computer program product.

[0005] The technical solution of this application is implemented as follows: This application provides an interactive control method, including: Get the first data of each object in the interaction object; Based on the first data of the interactive objects, a first feature of each of the objects and a second feature between the interactive objects are determined; wherein, the first feature is used to characterize one or more of the following: shape information, posture information, position information and category information; the second feature is used to characterize spatial motion change information between the interactive objects; The second data, which is a compression of the first feature and the second feature, is sent to the second end, so that the second end can obtain the first feature, the second feature and the third feature based on the second data, and determine the offset information of the target object based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

[0006] In the above scheme, the interactive objects include a first object and a second object; the first object interactively controls the movement of the second object; the first feature includes: a first sub-feature and a second sub-feature; the first data is used to characterize the position information of multiple points of the corresponding object; the step of determining the first feature of each object and the second feature between the interactive objects based on the first data includes: Based on the first data of the first object, a first sub-feature of the first object is determined; wherein, the first sub-feature is used to characterize the shape information and the posture information of the first object; Based on the first data of the second object, a second sub-feature of the second object is determined; wherein, the second sub-feature is used to characterize the location information of the predetermined point of the second object and the category information; The second feature is determined by performing feature fusion processing on the first data corresponding to the first object and the second object respectively.

[0007] In the above scheme, the step of performing feature fusion processing based on the first data corresponding to the first object and the second object respectively to determine the second feature includes: Based on the first preset model, process the first data of the first object and the first data of the second object to determine the fourth feature of each point corresponding to the first object and the fifth feature of each point corresponding to the second object. The second feature is determined by fusing the fourth and fifth features based on a cross-modal attention mechanism.

[0008] This application also provides an interactive control method, including: Receive the second data sent by the first end; Wherein, the second data is determined by the first end by compressing the first feature and the second feature; the first feature and the second feature are determined by the first end based on the first data of the interactive object; the first feature is used to characterize one or more of the following of the corresponding object: shape information, posture information, position information and category information; the second feature is used to characterize the spatial motion change information between interactive objects; Decompress the second data to obtain the first feature and the second feature; A third feature is extracted based on the first feature and the second feature, and the offset information of the target object is determined based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

[0009] In the above scheme, the interaction objects include a first object and a second object; the first object interacts to control the movement of the second object; the first feature includes: a first sub-feature and a second sub-feature; the decompression of the second data to obtain the first feature and the second feature includes: Decompressing the second data yields the first sub-feature corresponding to the first object, the second sub-feature corresponding to the second object, and the second feature; The first sub-feature is used to characterize the shape information and the posture information of the first object; the second sub-feature is used to characterize the position information and the category information of the predetermined point of the second object.

[0010] In the above scheme, the step of extracting a third feature based on the first feature and the second feature, and determining the offset information of the target object based on the first feature, the second feature, and the third feature, includes: First data of the first object is restored based on the first sub-feature, and first data of the second object is restored based on the second sub-feature; wherein, the first data is used to characterize the position information of multiple points of the corresponding object; Based on the first data corresponding to the first object and the second object respectively, determine the N first points in the first object that are closest to the second object; N is an integer greater than 0; The third feature is determined based on N first points; The offset information is determined by processing the first sub-feature, the second sub-feature, the second feature, and the third feature based on the second preset model.

[0011] The method in the above scheme further includes: Based on the category information, the initial position information of each point corresponding to the second object is determined in a preset information set; wherein, the preset information set includes multiple categories and the initial position information of each point of the corresponding object.

[0012] Based on the offset information and the initial position information of each point corresponding to the second object, the target object corresponding to the second object is controlled to offset.

[0013] This application embodiment also provides a first electronic device, including a first memory, a first processor and a first communication interface. The first memory stores a computer program that can run on the first processor and the first communication interface. When the first processor executes the computer program, it implements the steps in the method described in the first application. When the first communication interface executes the computer program, it implements the steps in the method of the first end.

[0014] This application also provides a second electronic device, including a second memory, a second processor, and a second communication interface. The second memory stores a computer program that can run on the second processor and the second communication interface. When the second processor executes the computer program, it implements the steps in the method described in the second application. When the second communication interface executes the computer program, it implements the steps in the method described in the second end.

[0015] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a first processor, implements the steps in the method described in the first application.

[0016] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a second processor, implements the steps in the method described in the second application.

[0017] This application also provides a computer program product, including a computer program that, when executed by a first processor, implements the steps in the method described in the first application.

[0018] This application also provides a computer program product, including a computer program that, when executed by a second processor, implements the steps in the method described in the second application.

[0019] In this embodiment, first data of each object in the interactive object is acquired; based on the first data of the interactive object, a first feature of each object and a second feature between the interactive objects are determined; wherein, the first feature is used to characterize one or more of the following: shape information, posture information, position information, and category information; the second feature is used to characterize the spatial action change information between the interactive objects; the second data after compressing the first and second features is sent to the second end, so that the second end can obtain the first, second, and third features based on the second data, and determine the offset information of the target object based on the first, second, and third features; wherein, the third feature is used to characterize the interaction details between the interactive objects. In this way, by acquiring the first data of each object in the interactive object, the first feature describing the object's own attributes and the second feature describing the relationship between the objects can be extracted; combined with the compression of the first and second features before transmission, the key interaction semantics can be preserved while reducing the amount of data, thereby improving transmission efficiency and reducing bandwidth consumption, and thus improving the accuracy of interaction perception. At the same time, the second end can further extract the third feature characterizing the interaction details and combine it with the existing features to jointly calculate the offset information, thereby achieving high-precision restoration of the interaction behavior and solving the problem of insufficient interaction representation accuracy in the prior art. Attached Figure Description

[0020] Figure 1 Flowchart of the interactive control method provided in the embodiments of this application Figure 1 ; Figure 2 Flowchart of the interactive control method provided in the embodiments of this application Figure 2 ; Figure 3 Flowchart of the interactive control method provided in the embodiments of this application Figure 3 ; Figure 4 This is a schematic diagram illustrating the effect of the interactive control method provided in the embodiments of this application; Figure 5 Flowchart of the interactive control method provided in the embodiments of this application Figure 4 ; Figure 6 Flowchart of the interactive control method provided in the embodiments of this application Figure 5 ; Figure 7 Flowchart of the interactive control method provided in the embodiments of this application Figure 6 ; Figure 8 Flowchart of the interactive control method provided in the embodiments of this application Figure 7 ; Figure 9 This application provides an example of the interactive control method's effects. Figure 9 ; Figure 10 An interactive schematic diagram of the interactive control method provided in the embodiments of this application; Figure 11 Schematic diagram of the interactive control device provided in the embodiments of this application Figure 1 ; Figure 12 A schematic diagram of a hardware entity of the first electronic device provided in an embodiment of this application; Figure 13 Schematic diagram of the interactive control device provided in the embodiments of this application Figure 2 ; Figure 14 This is a schematic diagram of a hardware entity of a second electronic device provided in an embodiment of this application.

[0021] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows: 1) Interaction Object: In this application, an interaction object refers to an entity participating in the human-object interaction process, including but not limited to human bodies (such as operators) and objects (such as tools, equipment, animals, machinery, etc.). Interaction objects interact with each other through some means of action or control relationship, such as a human body controlling the movement of an object.

[0024] 2) First data: This refers to the point cloud data of each object in the interactive object, used to represent the positional information of multiple points of the object. This point cloud data reflects the spatial distribution and geometric structure of the object and is the basis for subsequent feature extraction.

[0025] 3) First feature: Attribute information used to characterize a single interactive object, which may include one or more of the following: shape information, pose information, position information, and category information. The first feature is an abstract representation of the first data, which helps to reduce data dimensionality and retain key semantic information.

[0026] 4) Second feature: Used to characterize spatial motion changes between interactive objects. This feature is obtained after feature fusion processing based on the first data of two or more interactive objects, reflecting the overall dynamic characteristics of the interaction between objects.

[0027] 5) Third feature: Used to characterize the interaction details between interactive objects. The third feature usually needs to be further extracted at the decoding end based on the first and second features to capture more refined interaction behaviors or contact relationships.

[0028] 6) Compression ratio: This refers to the proportional parameter used in the compression process to control the size of the compressed data. The compression ratio can be adjusted according to the user's needs for the transmission bitstream, thereby achieving a balance between data accuracy and transmission efficiency.

[0029] 7) Cross-modal attention mechanism: A deep learning model architecture used to fuse information from different modalities (such as human body and object). In this application, the cross-modal attention mechanism is used to extract global interaction features between interactive objects, enhancing the expressive power of interaction modeling.

[0030] 8) Skinned Multi-Person Linear Model (SMPL): This is a parametric 3D human pose model that drives high-fidelity human mesh generation through low-dimensional parameters (such as pose θ and shape β). Its core advantage lies in simplifying complex skeletal animation into linear blended skinning (LBS) and pose correction deformation, supporting efficient reconstruction of human pose from monocular RGB images or point clouds. In HOI scenarios, SMPL can be used to decouple human actions from object interactions, providing a standardized human representation foundation for the "human-object parameter layer".

[0031] 9) Huffman coding: Huffman coding is a lossless compression algorithm based on statistical probability. It minimizes data entropy by assigning short codewords to high-frequency symbols and long codewords to low-frequency symbols. Its core steps include frequency statistics, constructing an optimal binary tree, and generating a prefix code table. In this invention, Huffman coding can be used to compress discrete parameters of interactive feature layers (such as contact region labels), and combined with arithmetic coding, it further improves semantic preservation capabilities at low bit rates.

[0032] 10) K-Nearest Neighbor (KNN) Search Strategy: The K-nearest neighbor search strategy is used to find the nearest points to the target point in point cloud data. In this application, KNN is used to extract the human point cloud points closest to the object's center to enhance the extraction effect of interactive semantic information.

[0033] 11) Point Cloud: A three-dimensional dataset composed of a large number of points, each containing coordinate information, used to represent the surface or structure of an object in three-dimensional space. In this application, point cloud is used to represent the geometric shape and position information of interactive objects.

[0034] 12) Template Method: A method based on a pre-defined standard model, used in this application to reconstruct the object's point cloud point cloud point by point based on the object's standard point cloud position and offset information. The template method helps improve reconstruction accuracy, especially in maintaining interactive quality under low bitrate conditions.

[0035] This application provides an interactive control method; please refer to [link / reference]. Figure 1 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 1 , will combine Figure 1 The steps shown are explained below: S101. Obtain the first data of each interactive object in the interactive objects.

[0036] In this embodiment, the interactive objects include entities participating in the human-object interaction process, such as the human body (operator) and objects (tools, equipment, etc.). The first data includes point cloud data for each interactive object, reflecting its spatial distribution and geometric structure. A point cloud is a three-dimensional dataset composed of a large number of points, each containing coordinate information to represent the surface or structure of an object in three-dimensional space. Therefore, the first data can be seen as a digital representation of the specific location and form of the interactive object in three-dimensional space. The first end can acquire the first data for each object through a sensor or camera.

[0037] For example, in a remote surgery scenario, the doctor interacts as a person with the surgical instruments as objects. The first end uses 3D sensors (such as depth cameras and LiDAR) to collect initial data on the doctor's hands and the surgical instruments.

[0038] Here, the first end refers to the encoding end, which is the end that acquires the first data of the interactive object. This first data is used to represent the positional information of multiple points corresponding to the object.

[0039] S102. Based on the first data of the interactive objects, determine the first feature of each interactive object and the second feature between the interactive objects; wherein, the first feature is used to characterize one or more of the following: shape information, posture information, position information and category information; the second feature is used to characterize the spatial motion change information between the interactive objects.

[0040] In this embodiment, the first feature is an abstract representation of a single interactive object, including but not limited to shape information, pose information, position information, and category information. This information can be used to describe the key attributes of the interactive object. For example, for a human body, the first end can process the first data using the SMPL model to extract the shape parameters (β) and pose parameters (θ) of the human body to describe the overall shape and joint posture of the human body; while for an object, the first end can extract the center coordinates and category label of the object to represent the position and type of the object in space.

[0041] In this embodiment, the second feature is used to characterize spatial motion change information between interactive objects. This feature is generated after feature fusion processing, based on the first data of two or more interactive objects, to obtain the first features of each object. The second feature can capture dynamically changing information during the interaction, such as the relative motion and contact relationship between the human body and objects.

[0042] In this embodiment of the application, by extracting the first feature and the second feature, the first end can retain key interaction information at a lower data dimension, which not only improves compression efficiency, but also enhances the ability to understand and reconstruct interaction details.

[0043] S103. The second data after compressing the first feature and the second feature is sent to the second end, so that the second end can obtain the first feature, the second feature and the third feature based on the second data, and determine the offset information of the target object based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

[0044] In this embodiment, the second data refers to the bitstream data after compression processing based on the first and second features. To achieve efficient transmission, the first end can use a lossless compression method—Huffman coding. After receiving the second data, the second end decodes it to obtain the first and second features. Then, it uses the point cloud data restored from the first feature to calculate the nearest neighbor point cloud and determine the third feature. The second feature is then fused with the first, second, and third features to determine the offset information.

[0045] The third feature refers to interaction-local features, which describe finer-grained contact relationships or local action patterns between interactive objects. For example, the K-nearest neighbor algorithm can be used to find the nearest points between human point clouds and object point clouds, and then the distance change trends of these nearest points can be analyzed to extract interaction-local features. These features can supplement global features and further improve the accuracy of interactive behavior recognition and reconstruction.

[0046] The second end is the decoding end, which receives and parses the second data, extracts the first and second features, and further extracts the third feature.

[0047] In this embodiment, the first end can employ a hierarchical compression strategy, which can significantly reduce transmission and storage overhead while ensuring data accuracy. The second end restores the object's offset through feature fusion, avoiding registration difficulties during rotation and translation, and obtains more accurate object shape and pose estimates through adaptive learning. Finally, using the object offset information output from the second end, the offset of each point in the object point cloud is adjusted based on a template method, thereby achieving high-precision reconstruction of the object point cloud.

[0048] In this embodiment, by acquiring the first data of each object in the interactive object, a first feature describing the object's own attributes and a second feature describing the relationship between objects can be extracted. Combining the compression of the first and second features before transmission reduces data volume while preserving key interactive semantics, thereby improving transmission efficiency and reducing bandwidth usage, and ultimately enhancing the accuracy of interactive perception. Simultaneously, at the second end, a third feature representing the interaction details can be further extracted and combined with existing features to calculate offset information, achieving high-precision reconstruction of the interactive behavior and solving the problem of insufficient interaction representation accuracy in existing technologies.

[0049] Please see Figure 2 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 2 , Figure 1 S102 in the process can also be implemented through S201 to S203, combining Figure 2 The steps shown are explained below: S201. Based on the first data of the first object, determine the first sub-feature of the first object; wherein the first sub-feature is used to characterize the shape information and posture information of the first object.

[0050] In this embodiment, the first data includes point cloud data of a first object (person), which typically contains geometric information such as spatial coordinates and normal vectors. The first end processes the point cloud data to extract the key features of the first object—the first sub-features. The first sub-features mainly describe the external contour (shape information) and dynamic parameters (posture information) of the first object, such as joint angles and limb orientation.

[0051] In this application, the extraction of the first sub-feature is typically achieved using the SMPL model. The SMPL model represents human morphology and pose in a low-dimensional parametric manner, thereby achieving efficient and accurate modeling. The use of the SMPL model not only reduces data dimensionality but also ensures reconstruction quality, making it suitable for subsequent encoding and transmission processes.

[0052] S202. Based on the first data of the second object, determine the second sub-feature of the second object; wherein the second sub-feature is used to characterize the location information and category information of the predetermined point of the second object.

[0053] In this embodiment of the application, the second sub-feature is used to describe the key attributes of the second object (i.e., the controlled object), mainly including the position (predetermined position) of certain important points on the surface of the second object and the category to which the second object belongs (such as tools, equipment, instruments, etc.). The first end can determine the position information of the predetermined position of the second object based on each point in the first data (point cloud data) of the second object, and determine the category information through the position of multiple points of the second object.

[0054] The predetermined points may include the center, highest point, lowest point, or a point on the outer contour of the second object.

[0055] S203. Based on the first data corresponding to the first object and the second object respectively, perform feature fusion processing to determine the second feature.

[0056] In this embodiment, the second feature is an abstract representation of the relationship between the first object and the second object, reflecting the spatial connection and interaction pattern between them. The first end can obtain the features of the first object and the second object respectively by performing feature transformation on the first data corresponding to the first object and the second object, and then perform feature fusion processing on the features of the first object and the second object. Cross-modal attention mechanism is usually used to capture the global interaction relationship to obtain the second feature.

[0057] In this embodiment, the first feature involves inputting the point cloud data of the human body and the object into a PointNet-based HOI encoder to fuse multi-scale features of the human body and the object. This encoder is implemented based on a specific PointNetv2, with the input being the specific point cloud and the output being the features corresponding to the human body and the object, respectively. Then, a cross-modal attention mechanism is introduced for global interaction modeling, where the human body feature is Q, and the object features are K and V. The output is a fused global interaction feature of the human body and the object (the second feature). Through this mechanism, we capture the correlation and interaction information between the human body and the object.

[0058] In this embodiment, the first object and the second object are decoupled into a first object and a second object, and feature vectors of the first object and the second object are extracted respectively. Then, cross-modal fusion technology is used to construct interactive features. This method can more finely depict the interaction relationship between people and objects, thereby improving the accuracy of interaction reconstruction. It can maintain a high-quality interactive representation of the interaction relationship between people and objects even at low bit rates, meeting the needs of high-precision application scenarios such as telemedicine and industrial remote operation.

[0059] In some embodiments, S203 can also be implemented by including S2031 to S2032, which will be described in conjunction with the steps: S2031: Based on the first preset model, process the first data of the first object and the first data of the second object to determine the fourth feature of each point corresponding to the first object and the fifth feature of each point corresponding to the second object.

[0060] In this embodiment, the first preset model refers to a deep neural network model used to extract local features from the interactive point cloud, typically implemented using the PointNetv2 architecture. The first preset model can extract features point-by-point from the first data of the input first object and second object, and output a high-dimensional semantic vector to obtain the fourth feature of the first object and the fifth feature of the second object. The fourth feature refers to the high-dimensional feature vector extracted from the point cloud of the first object (such as a human body) using the first preset model. It represents the local geometric attributes and interaction context information of each point of the first object (such as a human body) in space. These features can be used to describe dynamic information such as contact relationships and posture changes between people and objects.

[0061] The fifth feature refers to the high-dimensional feature vector extracted from the point cloud of the second object (such as a physical object) using the first preset model. This high-dimensional feature vector is used to express the shape, texture, and interaction state of the object at various points in space. By comparing and fusing the fifth feature with the fourth feature, the system can more accurately capture the interaction patterns between people and objects.

[0062] In this embodiment of the application, the features of the first object and the second object are extracted by the first preset model, which can retain the key details of human-object interaction, while removing redundant information and improving the efficiency and accuracy of subsequent feature fusion processing.

[0063] S2032. Based on the cross-modal attention mechanism, the fourth feature and the fifth feature are fused to determine the second feature.

[0064] In this embodiment, the first end can fuse the fourth feature as the Q feature of the cross-modal attention mechanism model and the fifth feature as the K and V features of the cross-modal attention mechanism model to obtain the second feature. Here, Q represents the query feature, K represents the keyword feature, and V represents the value feature.

[0065] The second feature is a comprehensive feature vector fused through a cross-modal attention mechanism, used to represent the global interaction relationship between people and objects. This second feature not only contains point cloud-level geometric information but also incorporates the temporal and spatial context of the interaction behavior, enabling the decoding end to more accurately reconstruct the interaction scene.

[0066] By fusing the fourth and fifth features through a cross-modal attention mechanism, complex interaction patterns between people and objects can be effectively captured, especially maintaining the integrity of the interaction semantics under low bit rate transmission. This fusion method allows for more accurate reconstruction of interaction actions at the decoding end, thereby improving compression efficiency and reconstruction quality.

[0067] In this embodiment, a first preset model is used to model the point cloud data of people and objects point by point, extracting the fourth and fifth features while retaining key interaction details. Subsequently, a cross-modal attention mechanism is used to fuse the fourth and fifth features in the point cloud data of people and objects to generate the second feature, thereby forming a high-dimensional abstraction of the overall interaction state. By adopting the above bottom-up modeling approach, the integrity and accuracy of the interaction semantics can still be maintained at low bit rates.

[0068] Please see Figure 3 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 3 , Figure 1 S103 in the above can also be implemented through S301, combining Figure 3 The steps shown are explained below: S301. Compress the first feature and the second feature based on a predetermined compression ratio to obtain the second data, and send it to the second terminal; wherein, the predetermined compression ratio is determined based on the user's requirements for the bitstream size of the second data.

[0069] In this embodiment, the predetermined compression ratio refers to a compression ratio parameter preset according to the user's specific needs for the amount of transmitted data (i.e., bitstream size). The predetermined compression ratio is used to control the balance between the degree of information preservation and the compression intensity during the compression process. For example, in high-precision interactive scenarios such as remote surgery or industrial teleoperation, users may prefer to use a lower compression ratio to retain more details; while in scenarios with high real-time requirements such as live video streaming, users may prefer to use a higher compression ratio to reduce bandwidth consumption. The first end can use Huffman coding to compress the first feature and the second feature based on the predetermined compression ratio to obtain the second data.

[0070] In this embodiment, the first end can first normalize the input first and second features and map them to discrete symbols. By calculating the frequency distribution of the symbols, a compression tree is constructed using Huffman coding to generate the encoding. The compressed data is stored in binary form. During decoding, the original data is recovered through inverse decoding and inverse normalization operations, thereby recovering the features of the human body, objects, and interaction information. Here, human body parameters, object features, and interaction features are all represented by feature vectors, so this compression method is used for each, and they are compressed separately. Because a layered compression strategy is adopted, different layers do not affect each other.

[0071] In this embodiment, the bitstream size is controlled by adjusting the compression ratio. Specifically, we primarily adjust the amount of data in the SMPL parameter of the character features to obtain the interactive effects at different bitrates. Experimental results are shown below. Figure 4 As can be seen, this method maintains good interactive quality even at low bit rates, satisfying human visual needs while providing necessary prior knowledge for downstream HOI tasks. Since our objects are predicted based on people, inaccurate person poses at low bit rates lead to significant deviations in object prediction. It is worth noting that reconstruction quality decreases at relatively low bit rates; however, the experimental results achieve a balance between subjective decoding quality and bitstream size up to approximately 0.5 bpp.

[0072] In this embodiment, by dynamically adjusting the predetermined compression ratio based on the user's demand for bitrate, different application scenarios can be flexibly adapted while ensuring interaction quality. This method of dynamically adjusting the predetermined compression ratio based on the user's demand for bitrate effectively controls data transmission overhead, thereby achieving high-quality interactive representation at low bitrates. This supports practical applications with high requirements for real-time performance and accuracy, such as telemedicine and live sports broadcasts.

[0073] Please see Figure 5 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 5 Fourth, it will be combined Figure 5 The steps shown are explained below: S401, Receive the second data sent by the first end.

[0074] Wherein, the second data is determined by the first end compressing the first feature and the second feature; the first feature and the second feature are determined by the first end based on the first data of the interactive object; the first data is used to characterize the position information of multiple points of the corresponding object; the first feature is used to characterize one or more of the following of the corresponding object: shape information, posture information, position information and category information; the second feature is used to characterize the spatial motion change information between interactive objects; In this embodiment of the application, by compressing the first feature and the second feature into the second data, the integrity of the interactive semantics can be guaranteed, and the transmission bandwidth requirement can be significantly reduced, thereby improving system efficiency and making the system suitable for low bit rate scenarios.

[0075] S402. Decompress the second data to obtain the first feature and the second feature.

[0076] In this embodiment, after receiving the second data, the second end decompresses the second data to recover the original first and second features. The decompression process typically employs the inverse operation of Huffman coding, restoring the compressed binary data to discrete symbols, which are then mapped back to the original continuous feature vector. This decompression process ensures lossless data recovery and prevents the loss of crucial interactive information.

[0077] In this embodiment, the decompressed first and second features can be used for subsequent feature fusion and offset calculations, thereby achieving high-precision reconstruction of the interactive scene. The design of using the decompressed first and second features for subsequent feature fusion and offset calculations ensures that sufficient detail information is retained even at low bit rates, meeting the needs of high-precision applications such as remote surgery and industrial teleoperation.

[0078] S403. Extract a third feature based on the first feature and the second feature, and determine the offset information of the target object based on the first feature, the second feature and the third feature.

[0079] In this embodiment, the third feature is used to characterize the interaction details between interactive objects. The third feature can capture detailed features such as local contact relationships and surface interaction intensity between people and objects. The second end can recover the first data of the first object and the second object using the first feature, and perform nearest neighbor search using the first data corresponding to the first object and the second object respectively to determine the point closest to the second object. Then, feature transformation is performed on this point to obtain the third feature. The offset information of the target object is determined by fusing the first feature, the second feature, and the third feature.

[0080] In this embodiment, the second end extracts human point clouds and object point clouds based on the first feature, then uses a KNN search strategy to obtain the human point cloud closest to the object's center, and extracts interactive local features (third features) from this human point cloud. Next, the first feature, second feature, and third feature are fused and input into the decoder to output the object's point-by-point offset vector.

[0081] In this embodiment, the second end can use a standard template of the object as a reference, and move each point according to the offset vector to obtain a complete object point cloud. This template-based offset method avoids the error problems caused by rotation and translation in traditional registration methods, and improves the accuracy and robustness of reconstruction.

[0082] In this embodiment, efficient representation and accurate reconstruction of human-object interaction scenes are achieved through hierarchical compression and multimodal feature fusion. Hierarchical compression and multimodal feature fusion reduce the transmission bit rate while preserving key interactive semantic information, thus providing a high-quality interactive experience even in low-bandwidth environments. This approach can be widely applied in various fields such as telemedicine, virtual reality, and live sports broadcasting.

[0083] Please see Figure 6 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 5 , Figure 5 S402 to S403 can also be implemented through S501 to S505, combining Figure 6 The steps shown are explained below: S501. Decompress the second data to obtain the first sub-feature corresponding to the first object, the second sub-feature corresponding to the second object, and the second feature.

[0084] In this embodiment, the compressed second data is decompressed to extract three key pieces of information: a first sub-feature corresponding to the first object, a second sub-feature corresponding to the second object, and a second feature describing the interaction behavior. These features respectively characterize the structural information and interaction semantics between the person and the object. Specifically, the first sub-feature characterizes the shape information and posture information of the first object; the second sub-feature characterizes the position information and category information of the predetermined point of the second object.

[0085] The first object includes the human body. The first sub-feature describes the geometric shape and posture changes of the first object, including but not limited to skeletal angles, joint positions, and limb lengths. The second object includes objects that interact with the first object, such as surgical instruments, tools, and balls. The second sub-feature describes the location and category information of predetermined points on the second object. Predetermined points refer to key points on or inside the surface of the second object, such as the center point, contact point, and edge point. Information about these predetermined points helps the system accurately locate the interaction area and recover the spatial posture of the second object. Category information is used to identify the type of the second object so that the appropriate template or modeling method can be called during the reconstruction process.

[0086] In this embodiment, extracting the sub-features corresponding to the first object and the second object respectively can effectively improve the modeling accuracy and compression efficiency of human-object interaction scenarios. It can reduce the bitrate while ensuring interaction quality, thereby achieving more efficient transmission and storage. Furthermore, the system can be widely applied in multiple fields such as telemedicine, virtual reality, and industrial remote operation.

[0087] S502. Restore the first data of the first object based on the first sub-feature, and restore the first data of the second object based on the second sub-feature.

[0088] In this embodiment of the application, the second end can reconstruct the point cloud data of the first object, i.e., the first data, based on the first sub-feature. The second end can reconstruct the point cloud data of the second object, i.e., the second data, based on the second sub-feature.

[0089] S503. Based on the first data corresponding to the first object and the second object respectively, determine the N first points in the first object that are closest to the second object; N is an integer greater than 0.

[0090] In this embodiment of the application, the second end can use the K-Nearest Neighbors (KNN) algorithm to process the point cloud data corresponding to the first object and the second object respectively, so as to identify the point cloud data of the first object that is closest to the second object.

[0091] In this embodiment of the application, the second end determines the distance information between each point in the first object and each point in the second object based on the position information of each point in the first data of the first object and the position information of each point in the first data of the second object; based on the distance information between each point in the first object and each point in the second object, N first points are determined.

[0092] S504. Determine the third feature based on N first points.

[0093] In this embodiment, the third feature refers to a local interaction feature composed of N first points, used to describe the contact relationship and spatial distribution between the first object and the second object. The second end can process the N first points through a HOI encoder to obtain the third feature.

[0094] S505. Based on the second preset model, process the first sub-feature, the second sub-feature, the second feature, and the third feature to determine the offset information.

[0095] In this embodiment, the second preset model refers to a pre-trained deep neural network model, such as PointNetV2 or a variant of PointNetV2, used to fuse multimodal features and output the final offset vector. The second end processes the first sub-feature, the second sub-feature, the second feature, and the third feature through the pre-trained second preset model, and generates accurate offset information through attention mechanisms and nonlinear transformations.

[0096] In this embodiment, by extracting and fusing multimodal features of the human body, objects, and their interactions in a layered manner, precise positioning of key points in human-object interactions can be achieved, thereby obtaining high-precision offset information. By extracting and fusing multimodal features of the human body, objects, and their interactions in a layered manner, the accuracy of object pose during reconstruction can be improved, thereby better preserving the semantics of human-object interactions, and thus effectively supporting high-quality interaction reconstruction tasks under low bitrate conditions.

[0097] Please see Figure 7 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 6 , will combine Figure 7 The steps shown are explained below: S601. Based on the position information of each point in the first data of the first object, render the first virtual object corresponding to the first object.

[0098] In this embodiment, the first data includes point cloud data, used to represent the spatial distribution and geometric features of the first object. The position information of each point includes three-dimensional coordinates (x, y, z), used to describe the specific position of the first object in space. The second end can generate a corresponding virtual object model by inputting the point information of the first object into the rendering module; this model is the first virtual object.

[0099] For example, in a remote surgery scenario, the system acquires the hand movements of the first patient using a point cloud acquisition device. Based on the point cloud data acquired by the device, the system reconstructs a virtual model of the patient's hand, thereby enabling visualization and remote guidance of the surgical procedure.

[0100] S602. Based on the position information of each point in the first data of the second object, render the second virtual object corresponding to the second object.

[0101] In this embodiment of the application, the first data includes point cloud data, used to represent the spatial distribution and geometric features of the first object. The position information of each point includes three-dimensional coordinates (x, y, z), used to describe the specific position of the first object in space. The second end can generate a corresponding virtual object model by inputting the point information of the second object into the rendering module. This model is the second virtual object.

[0102] In this embodiment, the second end collects point cloud data of the first object and converts the system-collected data into a first virtual object to simulate the first object. Subsequently, point cloud data of the second object is collected to generate a second virtual object to represent the second object. The first and second virtual objects are synchronously rendered and updated in the same interactive environment to form a complete human-object interaction scene. The entire process realizes the digital expression of complex interactive behaviors, enabling users to obtain high-fidelity operation feedback and control capabilities in a remote environment.

[0103] Please see Figure 8 The above is a flowchart illustrating the interactive control method provided in the embodiments of this application. Figure 7 , will combine Figure 8 The steps shown are explained below: S701. Based on the category information, determine the initial position information of each point corresponding to the second object in the preset information set.

[0104] In this embodiment, category information refers to data labels used to identify object types. For example, the category information identifies an object as a surgical instrument, basketball, soccer ball, or other specific type of object. The second end, by identifying the category of the object identified by the category information, can call up a standard template point cloud associated with the object, thereby assisting in reconstructing the initial geometric structure of the object identified by the category information.

[0105] The preset information set is a pre-stored database containing standard point cloud templates for different categories of objects, along with their corresponding spatial distribution features. These standard point cloud templates can be used to quickly locate and initialize 3D models of objects. The preset information set can be updated and expanded according to actual application scenarios to support interactive modeling of more types of objects.

[0106] Initial position information refers to the original spatial coordinates of each point of the object identified by the category information before any interaction occurs. This initial position information represents the baseline state of the object before any human interaction and is crucial for subsequent offset calculations. By matching the category information with a preset information set, the system can quickly find the initial position distribution corresponding to the current object, thus providing a foundation for subsequent interaction modeling of each point corresponding to the second object.

[0107] S702. Based on the offset information and the initial position information of each point corresponding to the second object, control the target object corresponding to the second object to offset.

[0108] In this embodiment, the offset information can be a vector-like point offset or overall attitude transformation parameters. The second end can control the target object to offset itself based on the offset information and the initial position information of each point corresponding to the second object, through corresponding components.

[0109] The initial position information and offset information together determine the final form of the object identified by the category information. The initial position information provides a reference state of the object when it is not interacted with, while the offset information describes how the object changes during interaction.

[0110] In this embodiment, the second end performs point-by-point adjustment of the point cloud of the object identified by the category information based on the offset information. The control process may involve various geometric transformations such as translation, rotation, and scaling to ensure that the shape of the object identified by the category information is accurate after interaction.

[0111] In this embodiment, the initial position information of each point corresponding to the second object is determined based on category information and a preset information set, and then offset point by point using offset information. This method can efficiently and accurately restore the dynamic changes of the object identified by the category information during the interaction process. This method can achieve high-quality HOI reconstruction at low bitrates, improve the practicality and application scope of digital human interaction, and can be widely used in scenarios with high precision requirements such as telemedicine, live sports broadcasting, and industrial remote operation.

[0112] This proposal addresses the issue of insufficient precision in character interaction representation during character transmission in existing technologies by providing a generative hierarchical human-object interaction compression method. This method decouples human-object interaction into parameterized lightweight representations of the human body, objects, and interaction features. Based on the proposed representation, the complex interaction representation is decoupled into two layers: a human-object parameter layer describing the overall structure of the human body and objects (first feature), and an interaction feature layer specifically representing contact details (third feature) and spatial movement changes (second feature). Through this hierarchical compression and feature extraction at the encoding and decoding ends, accurate capture of interaction semantics is achieved, while retaining key information and eliminating redundancy during compression. The innovation lies in: 1. At the encoding end, human body parameters and object features are extracted based on the human body point cloud and object point cloud. Global interaction features between humans and objects are then extracted from the human body point cloud and object point cloud. These human body parameters, object features, and global interaction features are compressed and encoded before being transmitted to the decoding end. This hierarchical compression scheme allows for flexible adjustments to different people, objects, and interaction information.

[0113] 2. On the decoding end, human point clouds and object point clouds are reconstructed using human parameters and object features respectively, thus realizing the reconstruction of human bodies and objects; human features are extracted based on human parameters, and interactive local features are extracted based on the human point cloud closest to the object's center; feature fusion processing is performed based on human features, object features, global interactive features, and local interactive features to reconstruct the object's offset; based on the object's offset and the template (initial point cloud position) of the object in the dataset, the object's point cloud is moved point by point to achieve accurate object point cloud acquisition and improve the representation accuracy of human interaction.

[0114] To achieve efficient compression and accurate reconstruction of human-object interactions, this paper proposes a compression-friendly HOI representation strategy. The core idea is to decompose the complex human-object interaction structure into three independent but collaborative semantic layers: the human layer, the object layer, and the interaction layer. Through structural decoupling, we not only improve the generality and reconstructability of the representation but also provide greater flexibility for subsequent feature compression and decoding processes.

[0115] Combination Figure 9At the encoding end, geometric and semantic features of the human body and objects are extracted respectively, and a global modeling mechanism is introduced to capture cross-modal interaction relationships. In the decoding stage, we combine the spatial context information of local regions to further recover the offset information and interaction posture of objects, so as to achieve fine human-object reconstruction. The following sections will introduce in turn: (1) the multi-level feature extraction module at the encoding end; (2) the compression and transmission mechanism of the bitstream; (3) the reconstruction process and interaction synthesis strategy at the decoding end.

[0116] Combination Figure 9 At the encoding end, point cloud data corresponding to the human body and objects are acquired respectively. To efficiently model human-object interaction information, the point cloud data is decoupled into a human body layer, an object layer, and an interaction layer. The human body layer and the object layer encode geometric and semantic features respectively, while the interaction layer captures the relationship between the two. Human body modeling adopts SMPL, using shape (β) and pose (θ) parameters as human body features (first features) to express global shape and joint rotation, precisely controlling human body details. The object layer extracts object center and category information as object features (first features) to overcome the problems of point cloud sparsity, occlusion, and non-rigidity. In the interaction layer, we input the point clouds of the human body and objects separately into a PointNet-based HOI encoder to fuse multi-scale features of the human body and objects. This encoder is based on a specific PointNetv2 implementation; the input is a specific point cloud, and the output is the feature vector corresponding to that point cloud. Then, we introduce a cross-modal attention mechanism for global interaction modeling, where the human body feature is Q, the object features are K and V, and the output is a fused global interaction feature of the human and object (second feature). Through this mechanism, we achieve the capture of the correlation and interaction information between the human and object.

[0117] Combination Figure 9 By using a decoupled representation method, we can accurately represent interactive point clouds based solely on human body features, object features, and global interaction features. To improve storage and transmission efficiency, we employ a lossless compression method—Huffman coding. The specific process is as follows: First, the input continuous data (such as global interaction features) is normalized and mapped to discrete symbols. By calculating the frequency distribution of the symbols, Huffman coding is used to construct a compression tree to generate the code. The compressed data is stored in binary form. During decoding, the original data is recovered through inverse decoding and inverse normalization operations, thereby recovering the features of the human body, objects, and interaction information. Here, human body parameters, object features, and interaction features are all represented in the form of feature vectors, so this compression method is used for each, and they are compressed separately. Because a layered compression strategy is adopted, different layers do not affect each other.

[0118] Combination Figure 9At the decoding end, we reconstruct human point cloud data from human features using a decoder to achieve human reconstruction; and reconstruct object point cloud data based on object features to achieve object reconstruction. We then utilize PointNet to extract human features from the human point cloud. We consider the human point closest to the object to be the most important part of the interaction; in fact, this part of the point data also contains the richest semantic information. Therefore, we choose the KNN search strategy to extract the human point cloud points closest to the object's center to enhance the interaction semantics. For this part of the local point cloud, we obtain the interaction local features (third features) using the same feature extractor as before. We also have object features and interaction global features transmitted from the encoding end. Next, we feed these multimodal features—human features, object features, global interaction features, and interaction local features—into a PointNetV2-based decoder, and reconstruct the object's offset through feature fusion. The input to this step is these four features, and the output is the point-by-point offset information of the object's point cloud. We describe the movement of the point cloud through the object's offset, avoiding the difficulties of rotation and translation registration, and use adaptive learning for more accurate object shape and pose estimation. Despite the large amount of offset data, layered compression is employed to ensure a balance between low bitrate and high-fidelity reconstruction. Finally, using the output object offset (referring to the offset of each point in the object's point cloud), we employ a template-based method to move the object's point cloud point by point according to the object's template (standard point cloud, the standard position of each point in the object's template in the dataset), thus achieving accurate object point cloud acquisition. The reconstruction of the person is based on the output of the SMPL decoder at the decoding end.

[0119] Please see Figure 10 The diagram below illustrates the interactive control method provided in this application, and the steps will be described in conjunction with the following: S801, The first end obtains the first data of each interactive object in the interactive object.

[0120] Alternatively, the implementation steps of S801 in this application embodiment can be referred to S101, and will not be described in detail here.

[0121] S802, the first end determines a first feature of each interactive object and a second feature between interactive objects based on the first data of the interactive objects; wherein, the first feature is used to characterize one or more of the following: shape information, posture information, position information and category information; the second feature is used to characterize spatial motion change information between interactive objects.

[0122] Alternatively, the implementation steps of S802 in this application embodiment can be referred to S102, and will not be described in detail here.

[0123] S803, the first end sends the second data after compressing the first feature and the second feature to the second end, so that the second end can obtain the first feature, the second feature and the third feature based on the second data, and determine the offset information of the target object based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

[0124] Alternatively, the implementation steps of S803 in this application embodiment can be referred to S103, and will not be described in detail here.

[0125] Please see Figure 11 A schematic diagram of the structure of the interactive control device provided in the embodiments of this application. Figure 1 .

[0126] This application also provides an interactive control device 600, including: an acquisition unit 601, a first determination unit 602, and a sending unit 603.

[0127] Acquisition unit 601 is used to acquire the first data of each object in the interactive object; The first determining unit 602 is configured to determine a first feature of each of the interactive objects and a second feature between the interactive objects based on the first data of the interactive objects; wherein the first feature is used to characterize one or more of the following: shape information, posture information, position information and category information; and the second feature is used to characterize spatial motion change information between the interactive objects. The sending unit 603 is used to send the second data after compressing the first feature and the second feature to the second end, so that the second end can obtain the first feature, the second feature and the third feature based on the second data, and determine the offset information of the target object based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

[0128] In this embodiment of the application, the interactive objects include a first object and a second object; the first object interactively controls the movement of the second object; the first feature includes: a first sub-feature and a second sub-feature; wherein, the first data is used to characterize the position information of multiple points of the corresponding object; the first determining unit 602 in the interactive control device 600 is used to determine the first sub-feature of the first object based on the first data of the first object; wherein, the first sub-feature is used to characterize the shape information and the posture information of the first object; Based on the first data of the second object, the second sub-feature of the second object is determined; wherein, the second sub-feature is used to characterize the location information of the predetermined point of the second object and the category information; The second feature is determined by performing feature fusion processing on the first data corresponding to the first object and the second object respectively.

[0129] In this embodiment of the application, the first determining unit 602 in the interactive control device 600 is used to process the first data of the first object and the first data of the second object based on the first preset model, and determine the fourth feature of each point corresponding to the first object and the fifth feature of each point corresponding to the second object. The second feature is determined by fusing the fourth and fifth features based on a cross-modal attention mechanism.

[0130] In this embodiment of the application, the sending unit 603 in the interactive control device 600 is used to compress the first feature and the second feature based on a predetermined compression ratio to obtain the second data, and send it to the second end; wherein, the predetermined compression ratio is determined based on the user's requirements for the bitstream size of the second data.

[0131] It should be noted that, in the embodiments of this application, if the above-described interactive control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an interactive control device (which may be a personal computer, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0132] Correspondingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps in the method on the first end.

[0133] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0134] It should be noted that, Figure 12 A hardware entity schematic diagram of the first electronic device provided in the embodiments of this application, such as... Figure 12As shown, this application embodiment provides a first electronic device 700, including a first memory 702, a first processor 701, and a first communication interface 703. The first memory 702 stores a computer program that can run on the first processor 701. When the first processor 701 executes the program, it implements the steps in the above-described method. When the first communication interface 703 executes the program, it implements the steps in the above-described method. The first processor 701 typically controls the overall operation of the first electronic device 700.

[0135] The first memory 702 is configured to store instructions and applications executable by the first processor 701, and can also cache data to be processed or already processed by the first processor 701 and the various modules in the first electronic device 700 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0136] Correspondingly, this application embodiment also provides a computer program product, including a computer program that can be executed by a first processor 701 of a first electronic device 700 to complete the steps in the first end-side method.

[0137] Please see Figure 13 A schematic diagram of the structure of the interactive control device provided in the embodiments of this application. Figure 2 .

[0138] This application embodiment also provides an interactive control device 800, including: a receiving unit 801, a decompression unit 802, and a second determining unit 803.

[0139] The receiving unit 801 is used to receive the second data sent by the first end; Wherein, the second data is determined by the first end by compressing the first feature and the second feature; the first feature and the second feature are determined by the first end based on the first data of the interactive object; the first feature is used to characterize one or more of the following of the corresponding object: shape information, posture information, position information and category information; the second feature is used to characterize the spatial motion change information between interactive objects; Decompression unit 802 is used to decompress the second data to obtain the first feature and the second feature; The second determining unit 803 is used to extract a third feature based on the first feature and the second feature, and to determine the offset information of the target object based on the first feature, the second feature and the third feature; wherein the third feature is used to characterize the interaction details between interactive objects.

[0140] In this embodiment of the application, the interactive object includes a first object and a second object; the first object interactively controls the movement of the second object; the first feature includes: a first sub-feature and a second sub-feature; the decompression unit 802 in the interactive control device 800 is used to decompress the second data to obtain the first sub-feature corresponding to the first object, the second sub-feature corresponding to the second object, and the second feature; The first sub-feature is used to characterize the shape information and the posture information of the first object; the second sub-feature is used to characterize the position information and the category information of the predetermined point of the second object.

[0141] In this embodiment of the application, the second determining unit 803 in the interactive control device 800 is used to restore the first data of the first object based on the first sub-feature, and to restore the first data of the second object based on the second sub-feature; the first data is used to characterize the location information of multiple points of the corresponding object. Based on the first data corresponding to the first object and the second object respectively, determine the N first points in the first object that are closest to the second object; N is an integer greater than 0; The third feature is determined based on N first points; The offset information is determined by processing the first sub-feature, the second sub-feature, the second feature, and the third feature based on the second preset model.

[0142] In this embodiment of the application, the interactive control device 800 is used to render the first virtual object corresponding to the first object based on the position information of each point in the first data of the first object; Based on the position information of each point in the first data of the second object, render the second virtual object corresponding to the second object.

[0143] In this embodiment of the application, the interactive control device 800 is used to determine the initial position information of each point corresponding to the second object in a preset information set based on the category information; wherein, the preset information set includes multiple categories and the initial position information of each point of the corresponding object.

[0144] Based on the offset information and the initial position information of each point corresponding to the second object, the target object corresponding to the second object is controlled to offset.

[0145] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a second processor, implements the steps in the method on the second end.

[0146] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0147] It should be noted that, Figure 14 This is a schematic diagram of a hardware entity of the second electronic device provided in an embodiment of this application, such as... Figure 14 As shown, this application embodiment provides a second electronic device 900, including a second memory 902, a second processor 901, and a second communication interface 903. The second memory 902 stores a computer program that can run on the second processor 901. When the second processor 901 executes the program, it implements the steps in the above-described method. When the second communication interface 903 executes the program, it implements the steps in the above-described method. The second processor 901 typically controls the overall operation of the second electronic device 900.

[0148] The second memory 902 is configured to store instructions and applications executable by the second processor 901, and can also cache data to be processed or already processed by the second processor 901 and the various modules in the second electronic device 900 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0149] Correspondingly, this application embodiment also provides a computer program product, including a computer program that can be executed by a second processor 901 of a second electronic device 900 to complete the steps in the second-side method.

[0150] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An interactive control method, characterized in that, include: Get the first data of each object in the interaction object; Based on the first data of the interactive objects, a first feature of each of the objects and a second feature between the interactive objects are determined; wherein, the first feature is used to characterize one or more of the following: shape information, posture information, position information and category information; the second feature is used to characterize spatial motion change information between the interactive objects; The second data, which is a compression of the first feature and the second feature, is sent to the second end, so that the second end can obtain the first feature, the second feature and the third feature based on the second data, and determine the offset information of the target object based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

2. The interactive control method according to claim 1, characterized in that, The interactive objects include a first object and a second object; the first object interactively controls the movement of the second object; the first feature includes a first sub-feature and a second sub-feature; the first data is used to characterize the position information of multiple points of the corresponding object; the step of determining the first feature of each object and the second feature between the interactive objects based on the first data includes: Based on the first data of the first object, the first sub-feature of the first object is determined; wherein, the first sub-feature is used to characterize the shape information and the posture information of the first object; Based on the first data of the second object, the second sub-feature of the second object is determined; wherein, the second sub-feature is used to characterize the location information of the predetermined point of the second object and the category information; The second feature is determined by performing feature fusion processing on the first data corresponding to the first object and the second object respectively.

3. The interactive control method according to claim 2, characterized in that, The step of performing feature fusion processing based on the first data corresponding to the first object and the second object respectively to determine the second feature includes: Based on the first preset model, process the first data of the first object and the first data of the second object to determine the fourth feature of each point corresponding to the first object and the fifth feature of each point corresponding to the second object. The second feature is determined by fusing the fourth and fifth features based on a cross-modal attention mechanism.

4. An interactive control method, characterized in that, include: Receive the second data sent by the first end; Wherein, the second data is determined by the first end by compressing the first feature and the second feature; the first feature and the second feature are determined by the first end based on the first data of the interactive object; the first feature is used to characterize one or more of the following of the corresponding object: shape information, posture information, position information and category information; the second feature is used to characterize the spatial motion change information between interactive objects; Decompress the second data to obtain the first feature and the second feature; A third feature is extracted based on the first feature and the second feature, and the offset information of the target object is determined based on the first feature, the second feature and the third feature; wherein, the third feature is used to characterize the interaction details between interactive objects.

5. The interactive control method according to claim 4, characterized in that, The interactive objects include a first object and a second object; the first object interactively controls the movement of the second object; the first feature includes a first sub-feature and a second sub-feature; the decompression of the second data to obtain the first feature and the second feature includes: Decompressing the second data yields the first sub-feature corresponding to the first object, the second sub-feature corresponding to the second object, and the second feature; Wherein, the first sub-feature is used to characterize the shape information and the posture information of the first object; the second sub-feature is used to characterize the position information of the predetermined point of the second object and the category information.

6. The interactive control method according to claim 5, characterized in that, The step of extracting a third feature based on the first feature and the second feature, and determining the offset information of the target object based on the first feature, the second feature, and the third feature, includes: First data of the first object is reconstructed based on the first sub-feature, and first data of the second object is reconstructed based on the second sub-feature; wherein, the first data is used to characterize the position information of multiple points of the corresponding object; Based on the first data corresponding to the first object and the second object respectively, determine the N first points in the first object that are closest to the second object; N is an integer greater than 0; The third feature is determined based on N first points; The offset information is determined by processing the first sub-feature, the second sub-feature, the second feature, and the third feature based on the second preset model.

7. The interactive control method according to any one of claims 4 to 6, characterized in that, The method further includes: Based on the category information, the initial position information of each point corresponding to the second object is determined in a preset information set; wherein, the preset information set includes multiple categories and the initial position information of each point of the corresponding object; Based on the offset information and the initial position information of each point corresponding to the second object, the target object corresponding to the second object is controlled to offset.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a communication interface. The memory stores a computer program that can run on the processor and the communication interface. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3, or implements the steps of the method according to any one of claims 4 to 7. When the communication interface executes the computer program, it implements the steps of the method according to any one of claims 1 to 3, or the steps of the method according to any one of claims 4 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3, or the steps of the method according to any one of claims 4 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3, or the steps of the method according to any one of claims 4 to 7.

Images