Heterogeneous hardware data collection method, apparatus, device, medium and program product

Abstract

The application provides a heterogeneous hardware data acquisition method and device, equipment, storage medium and program product, which can be applied to the technical field of robots. The heterogeneous hardware data acquisition method comprises the following steps: acquiring a first motion state parameter of a first type of actuator, the first motion state parameter being expressed in physical units; acquiring a second motion state parameter of a second type of actuator, the second motion state parameter being expressed in a numerical representation system; mapping the first motion state parameter from physical units to a target numerical interval according to a preconfigured unified data specification to obtain first standard format data; mapping the second motion state parameter from the numerical representation system to the same target numerical interval as that after the mapping of the first motion state parameter according to the preconfigured unified data specification to obtain second standard format data; and splicing the mapped first standard format data and the second standard format data into a single data structure in a predetermined order.

Description

Technical Field

[0001] This application relates to the field of robotics technology, specifically to a heterogeneous hardware data acquisition method, apparatus, device, medium, and program product. Background Technology

[0002] In the field of robotics, performing heterogeneous hardware data acquisition tasks typically requires the simultaneous use of multiple different types of actuators, including robotic arms, dexterous hands, teleoperated devices, and tactile sensors. These actuators come from different manufacturers and each employs different data output formats and encoding protocols. Robotic arms usually output joint state data in standard physical units, while dexterous hands or specialized actuators may use proprietary encoding protocols to represent motion state parameters using interval encoding or discretized numerical values. This heterogeneity in data formats prevents direct fusion and comparative analysis of data from different actuators. Constructing a unified training dataset for multiple actuators requires extensive manual data annotation and format conversion, reducing data processing efficiency and increasing the risk of errors. Summary of the Invention

[0003] In view of the above problems, this application provides heterogeneous hardware data acquisition methods, apparatus, devices, media and program products.

[0004] According to a first aspect of this application, a heterogeneous hardware data acquisition method is provided, comprising: acquiring a first motion state parameter of a first type of actuator, the first motion state parameter being represented in physical units; acquiring a second motion state parameter of a second type of actuator, the second motion state parameter being represented in a numerical representation system; mapping the first motion state parameter from the physical units to a target numerical range according to a pre-configured unified data specification to obtain first standard format data; mapping the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification to obtain second standard format data, wherein the unified data specification defines the mapping relationship from different physical units and numerical representation systems to a unified numerical range; and concatenating the mapped first standard format data and second standard format data in a predetermined order into a single data structure, wherein each position in the single data structure can uniquely determine the actuator type, left / right side, and joint number.

[0005] According to an embodiment of this application, the step of mapping the first motion state parameter from the physical unit to a target numerical range according to a pre-configured unified data specification to obtain first standard format data includes: obtaining the theoretical minimum and theoretical maximum values ​​of the first motion state parameter from the parameter configuration of the first type of actuator; and linearly mapping the first motion state parameter represented by the physical unit to the target numerical range according to the theoretical minimum and theoretical maximum values ​​obtained from the parameter configuration of the first type of actuator to obtain first standard format data.

[0006] According to embodiments of this application, the numerical representation system represents a physical unit representation or a numerical encoding representation. When the numerical representation system represents a physical unit representation, the step of mapping the second motion state parameter from the numerical representation system to a target numerical interval identical to the mapped first motion state parameter, according to the pre-configured unified data specification, to obtain second standard format data, includes: obtaining the theoretical minimum and maximum values ​​of the second motion state parameter from the parameter configuration of the second type of actuator; linearly mapping the motion state parameter represented by the numerical representation system to the target numerical interval based on the theoretical minimum and maximum values ​​obtained from the parameter configuration of the second type of actuator, to obtain second standard format data; When the numerical representation system represents a numerical encoding representation, the step of mapping the second motion state parameter from the numerical representation system to a target numerical interval identical to the mapped first motion state parameter, according to the pre-configured unified data specification, to obtain second standard format data includes: obtaining the lower and upper limits of the original quantization interval defined by the encoding protocol of the joint position of the second type of actuator from the unified data specification; linearly mapping the second motion state parameter represented by the numerical encoding to the target numerical interval based on the lower and upper limits of the original quantization interval.

[0007] According to embodiments of this application, the first type of actuator includes one of a robotic arm, a dexterous hand, and a teleoperated device; the second type of actuator includes another of a robotic arm, a dexterous hand, and a teleoperated device; the first motion state parameters include joint velocity and torque; the physical unit of the joint velocity of the first motion state parameter is expressed in radians per second; the physical unit of the torque of the first motion state parameter is expressed in Newton-meters; the second motion state parameters include joint velocity and torque; the numerical representation system of the joint velocity of the second motion state parameter is interval encoding; the numerical representation system of the torque of the second motion state parameter is Newton-meters. According to the pre-configured unified data specification, the first motion state parameter is mapped from the physical unit to the target numerical range to obtain the first standard format data. According to the pre-configured unified data specification, the second motion state parameter is mapped from the numerical representation system to the same target numerical range as the first motion state parameter after mapping to obtain the second standard format data. This includes: according to the unified data specification, converting the joint velocities of the first type of actuator and the second type of actuator into dimensionless relative velocity values, and converting the torques of the first type of actuator and the second type of actuator into percentage or normalized torque values.

[0008] According to an embodiment of this application, the step of splicing the mapped first standard format data and the second standard format data into a single data structure in a predetermined order includes: splicing the converted velocity and torque into a single data structure in a fixed order of left arm → right arm → left hand → right hand, which is the same as the joint position.

[0009] According to an embodiment of this application, the heterogeneous hardware data acquisition method further includes: acquiring raw state data of a third type of actuator, the third type of actuator including an array of tactile sensors, the raw state data including tactile matrix data with a single-finger resolution of 12×6 and the combined force value data of the five fingers, the tactile matrix data being stored in raw matrix form, and the combined force value data being in grams; according to the pre-configured unified data specification, flattening the tactile matrix data into a 72-dimensional vector in row-major order, converting each element in the 72-dimensional vector from the raw quantization range of 0-255 into a dimensionless normalized pressure value, converting the combined force value data from grams into a dimensionless normalized pressure value; and adding the converted tactile matrix data and the combined force value data as independent fields to the single data structure.

[0010] A second aspect of this application provides a heterogeneous hardware data acquisition device, comprising: a first acquisition module for acquiring a first motion state parameter of a first type of actuator, the first motion state parameter being represented in physical units; a second acquisition module for acquiring a second motion state parameter of a second type of actuator, the second motion state parameter being represented in a numerical representation system; a first mapping module for mapping the first motion state parameter from the physical units to a target numerical range according to a pre-configured unified data specification, to obtain first standard format data; a second mapping module for mapping the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain second standard format data, wherein the unified data specification defines the mapping relationship from different physical units and numerical representation systems to a unified numerical range; and a splicing module for splicing the mapped first standard format data and the second standard format data into a single data structure in a predetermined order, wherein each position in the single data structure can uniquely determine the actuator type, left / right side, and joint number.

[0011] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0012] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0013] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0014] According to some embodiments of this application, this application obtains motion state parameters of different types of actuators, and after uniformly mapping parameters of different physical units and numerical representation systems to the same target numerical range, splices them into a single data structure that can be uniquely traced, it realizes the unified and standardized processing and fusion of heterogeneous hardware data, overcomes the problem that data cannot be directly fused and compared due to inconsistent data formats between different types of actuators, and provides structured and standardized multi-actuator fusion input data for downstream machine learning models. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 The diagram illustrates an application scenario of the heterogeneous hardware data acquisition method according to an embodiment of this application.

[0017] Figure 2 A flowchart illustrating a heterogeneous hardware data acquisition method according to an embodiment of this application is shown schematically.

[0018] Figure 3 A schematic block diagram of a heterogeneous hardware data acquisition device according to an embodiment of this application is shown.

[0019] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a heterogeneous hardware data acquisition method according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0022] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0023] Figure 1 The diagram illustrates an application scenario of a heterogeneous hardware data acquisition method according to an embodiment of this application. For example... Figure 1As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send information, etc.

[0024] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal devices. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, and it can include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, and can also be in the form of web pages.

[0025] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and basic cloud computing services such as big data. The server can be the backend server of the aforementioned target application, used to provide backend services to the clients of the target application.

[0026] It should be noted that the heterogeneous hardware data acquisition method provided in this application embodiment can generally be executed by server 105 and / or first terminal device 101, second terminal device 102, and third terminal device 103. Accordingly, the heterogeneous hardware data acquisition device provided in this application embodiment can generally be disposed in server 105 and / or first terminal device 101, second terminal device 102, and third terminal device 103.

[0027] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0028] Figure 2 A flowchart illustrating a heterogeneous hardware data acquisition method according to an embodiment of this application is shown. Figure 2 As shown, the heterogeneous hardware data acquisition method according to the embodiments of this application may include steps S210 to S250.

[0029] In step S210, the first motion state parameter of the first type of actuator is obtained, and the first motion state parameter is expressed in physical units.

[0030] In some embodiments, the first type of actuator refers to an actuator device in a heterogeneous hardware data acquisition system that can directly output motion state parameters in standard physical units, including a six-degree-of-freedom industrial collaborative robotic arm, a precision positioning platform, etc. The first motion state parameters include at least the real-time position, joint velocity, and joint torque of each joint. These values ​​are expressed using standard physical units in the International System of Units (SI): joint position in radians, joint velocity in radians per second, and joint torque in Newton-meters. These actuators typically incorporate high-precision absolute encoders or torque sensors. The raw sensor readings have already been converted to physical units at the actuator firmware level, and the data acquisition system can directly read these values ​​with clear physical meaning via a standard fieldbus.

[0031] In some embodiments, the acquisition method is as follows: The data acquisition system communicates with the first type of actuator via a standard industrial bus, configures a fixed sampling period (e.g., 100Hz or 1kHz), and reads motion state data frames of each joint in batches according to a fixed order from the base joint to the end joint. The communication protocol stack performs verification and timing synchronization at the data link layer to ensure that multi-axis data is locked and reported on the same clock edge. The raw bus data read is directly parsed into standard physical unit values ​​by the driver or firmware interface, without requiring additional unit conversion logic from the acquisition system.

[0032] This embodiment obtains motion data with standard physical units at the data acquisition source by acquiring the first motion state parameters of the first type of actuator, providing a direct basis for subsequent heterogeneous data format unification and cross-actuator data fusion.

[0033] In step S220, the second motion state parameters of the second type of actuator are obtained, and the second motion state parameters are represented in a numerical representation system.

[0034] In some embodiments, the second type of actuator refers to an actuator device in a heterogeneous hardware data acquisition system whose motion state parameters are expressed in non-standard physical units or proprietary encoding protocols. These actuators typically come from different hardware vendors, each employing its own custom data encoding format. For example, the angles of each joint in a dexterous hand might be represented by 12-bit integer values, velocity by 8-bit integer values, and torque by 10-bit integer values. The numerical representation system can include physical unit representation (meters per second, etc.) and numerical encoding representation (discretizing continuous physical quantities into integer values ​​within a specific range using a preset encoding protocol).

[0035] In some embodiments, the acquisition method is as follows: The data acquisition system communicates with the second type of actuator through an application layer protocol and parses the raw data frame according to the communication protocol document provided by the actuator manufacturer. The raw data frame typically contains a frame header, device ID, data payload, and checksum field, where motion state parameters are stored in the data payload as raw encoded values ​​without physical conversion. The system temporarily stores these encoded values ​​and their corresponding encoding protocol identifiers in the raw data buffer, awaiting subsequent standardized mapping processing.

[0036] For example, a dexterous hand with 16 degrees of freedom reports the status of each joint via a serial port (baud rate 921600 bps) at a frequency of 500 Hz, with each frame consisting of 18 bytes. The protocol defines joint angles as 12-bit integers (0 to 4095 corresponding to the physical range of -90° to 90°), joint velocities as 8-bit integers (0 to 255 corresponding to 0 to 180° / s), and joint torques as 10-bit integers (0 to 1023 corresponding to 0 to 5 N·m). The data acquisition system parses the data byte by byte according to the protocol: joint angle encoding value 2048, joint velocity encoding value 128, and joint torque encoding value 511. These encoded values, along with the encoding protocol type (e.g., 'type=12bit_angle,8bit_vel,10bit_torque'), are stored in the raw data dictionary for subsequent mapping processing.

[0037] This embodiment supplements the data source output in the form of private encoding in the heterogeneous hardware data acquisition system by obtaining the second motion state parameters of the second type of actuator, and provides a complete set of input data for subsequently mapping parameters of different numerical representation systems to the same standardized numerical space.

[0038] In step S230, according to the pre-configured unified data specification, the first motion state parameter is mapped from physical units to the target numerical range to obtain the first standard format data.

[0039] In some embodiments, a unified data specification is a predefined set of standardized mapping rules used to uniformly convert motion state parameters from different physical units and numerical representation systems to standard format data within the same numerical range. The target numerical range refers to a standardized numerical range (e.g., [-1, 1] or [0, 1]), within which all motion state parameters of the same type from all actuators fall after mapping. After data normalization to a unified range, similar parameters from different sources can be numerically compared, statistically analyzed, and used for model training under the same dimensions, eliminating the incomparability of values ​​caused by differences in physical units.

[0040] In some embodiments, the mapping operation is based on the physical range limits (theoretical minimum and theoretical maximum) defined in the actuator parameter configuration, using a linear scaling formula: Mapped value = (Original value - Theoretical minimum) / (Theoretical maximum - Theoretical minimum) × (Upper limit of target interval - Lower limit of target interval) + Lower limit of target interval. This mapping method maintains the linear proportional relationship of the original data, meaning the relative size relationship between two points in the original data is completely preserved after mapping. When the original value exceeds the theoretical range, the mapping result will be truncated to the boundary value of the target value interval to prevent mapping out-of-bounds errors caused by abnormal data.

[0041] For example, the theoretical velocity range of a robotic arm joint is -3.14 rad / s to 3.14 rad / s, with a target value range of [-1, 1]. If the actual measured joint velocity is 1.57 rad / s, within the theoretical range, the mapped value is 1.57 / 3.14 = 0.5. If a value of -4.0 rad / s (exceeding the theoretical lower limit of -3.14) is measured due to momentary sensor interference, the mapped value is truncated to -1.0 to prevent abnormal data from affecting subsequent processing.

[0042] This embodiment eliminates the dimensional differences between different physical units by linearly mapping the first motion state parameter represented by physical units to the target numerical range, enabling motion state parameters of the same type but from different sources to be compared and jointly analyzed in a unified standardized numerical space.

[0043] In step S240, according to the pre-configured unified data specification, the second motion state parameter is mapped from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, to obtain the second standard format data. The unified data specification defines the mapping relationship from different physical units and numerical representation systems to the unified numerical range.

[0044] In some embodiments, the mapping rule for the second motion state parameter is based on the same unified data specification as the mapping rule for the first motion state parameter, ensuring that the same motion state parameter from different types of actuators falls into the exact same numerical range after mapping. For the second motion state parameter expressed in physical units, the same linear scaling formula as in step S230 is used, and the processing logic is completely consistent with that for the first type of actuator. For the second motion state parameter expressed in numerical encoding, the lower and upper limits of the original quantization range must first be obtained according to the encoding protocol, and then the encoded value is mapped to the target numerical range.

[0045] In some embodiments, the mapping formula for numerical coding parameters is: Mapped value = (Coded value - Lower quantization limit) / (Upper quantization limit - Lower quantization limit) × (Upper limit of target interval - Lower limit of target interval) + Lower limit of target interval. This mapping transforms the discretized coded values ​​into a standardized numerical range, while eliminating the differences in numerical distribution caused by different quantization resolutions between different coding protocols.

[0046] This embodiment achieves direct comparability and fusion of the same motion state parameters between heterogeneous actuators by mapping the second motion state parameter to the same target numerical range as the first motion state parameter, thus eliminating the numerical representation barrier for joint data analysis of multiple actuators.

[0047] In step S250, the mapped first standard format data and second standard format data are concatenated into a single data structure in a predetermined order, wherein each position in the single data structure can uniquely determine the actuator type, left and right sides, and joint number.

[0048] In some embodiments, the single data structure is a fixed-length one-dimensional floating-point array used to store standardized motion parameters from all actuator types in a predefined order. Each parameter value's index position in the array has a defined semantic meaning; from the index value, one can infer which actuator the data originates from, whether it belongs to the left or right side, which joint it corresponds to, and what parameter type it belongs to (position, velocity, or torque). This fixed layout design allows downstream machine learning models to directly utilize data position encoding as sensor source information without additional feature engineering.

[0049] In some embodiments, the splicing strategy adopts a grouping approach based on the physical layout of the actuators (e.g., left arm → right arm → left hand → right hand). Within the same actuator, the order of arranging joint positions, joint velocities, and joint torques is first determined, then the joint positions. Taking the left arm with 6 joints and the right hand with 16 degrees of freedom as an example, the dimension of a single data structure is (6+16)×3=66 dimensions. Indices 0-5 represent the normalized positions of the 6 joints in the left arm, indices 6-11 represent the normalized velocities of the 6 joints in the left arm, indices 12-17 represent the normalized torques of the 6 joints in the left arm, indices 18-33 represent the normalized positions of the 16 degrees of freedom in the right hand, and so on.

[0050] This embodiment provides downstream machine learning models with structured, standardized input data that encodes the source information of actuators by concatenating standard format data in a predetermined order into a single, uniquely traceable data structure. This enables the models to implicitly utilize sensor layout information for feature learning.

[0051] According to the heterogeneous hardware data acquisition method of this application, through the coordinated operation of steps S210 to S250, unified and standardized processing of motion state parameters of different physical units and numerical representation systems output by heterogeneous hardware devices is achieved, overcoming the problem that data cannot be directly fused and compared between different types of actuators due to inconsistent data formats. By splicing the standardized data into a uniquely traceable single data structure in a predetermined order, structured and standardized multi-actuator fusion input data is provided for downstream machine learning models.

[0052] According to some embodiments of this application, step S230 maps the first motion state parameter from physical units to the target numerical range according to the pre-configured unified data specification to obtain the first standard format data, including steps S231 to S232.

[0053] In step S231, the theoretical minimum and theoretical maximum values ​​of the first motion state parameters are obtained from the parameter configuration of the first type of actuator.

[0054] In some embodiments, the theoretical minimum value refers to the minimum physical value that the motion state parameter can reach (such as the negative limit of a joint angle), and the theoretical maximum value refers to the maximum physical value that can be reached (such as the positive limit of a joint angle). Both are obtained from the actuator's product specifications or configuration files and are usually recorded in physical units.

[0055] This embodiment obtains the theoretical minimum and maximum values ​​from the actuator parameter configuration, providing an accurate scaling benchmark for subsequent linear mapping calculations and ensuring that the mapping results are not distorted due to parameter range estimation deviations.

[0056] In step S232, based on the theoretical minimum and theoretical maximum values ​​obtained from the parameter configuration of the first type of actuator, the first motion state parameter represented by physical units is linearly mapped to the target numerical range to obtain the first standard format data.

[0057] In some embodiments, the linear mapping formula is: Mapped value = (Original value - Theoretical minimum value) / (Theoretical maximum value - Theoretical minimum value) × (Upper limit of target interval - Lower limit of target interval) + Lower limit of target interval. This mapping is a proportional scaling, maintaining the linear proportional relationship and distribution pattern of the original data.

[0058] For example, the theoretical speed range of a robotic arm joint is -2.0 rad / s to 2.0 rad / s, and the target value range is [-1, 1]. If the actual joint speed is 1.5 rad / s, within the theoretical range, the mapped value = (1.5 - (-2.0)) / (2.0 - (-2.0)) × 2 - 1 = 0.5.

[0059] This embodiment standardizes physical unit parameters to a target numerical range through linear mapping, providing a unified scaling benchmark for numerical alignment between heterogeneous actuators.

[0060] According to some embodiments of this application, the numerical representation system characterizes physical unit representations or numerical coding representations.

[0061] In the case where the numerical representation system represents the physical unit representation, the step of mapping the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain the second standard format data includes steps S241 to S242.

[0062] In step S241, the theoretical minimum and theoretical maximum values ​​of the second motion state parameters are obtained from the parameter configuration of the second type of actuator.

[0063] In some embodiments, some devices in the second type of actuator, although classified under a numerical representation system, still express their motion state parameters in physical units. However, the physical units used may be non-standard units (such as millimeters, percentage scales), or custom scales. For these parameters, the theoretical minimum and maximum values ​​of the motion state parameter need to be read from the actuator's technical specifications or parameter configuration file; that is, the physically achievable limits of the parameter, serving as the scaling reference for subsequent linear mapping. For example, the angle of a dexterous hand's finger joints, although expressed in millimeters (… The output is in units, but its physical range is still -90° to 90° (i.e. -90000 m° to 90000 m°), which is obtained directly from the actuator configuration.

[0064] This embodiment provides an accurate scaling reference for subsequent linear mapping by obtaining the theoretical minimum and maximum values ​​of the second motion state parameters expressed in physical units, enabling non-standard physical unit parameters to be processed with the parameters of the first type of actuator using a unified mapping logic.

[0065] In step S242, based on the theoretical minimum and maximum values ​​obtained from the parameter configuration of the second type of actuator, the motion state parameters represented by the numerical representation system are linearly mapped to the target numerical range to obtain the second standard format data.

[0066] In some embodiments, the second motion state parameter, expressed in physical units, is subjected to the same linear mapping formula as in step S232: Mapped value = (Original value - Theoretical minimum value) / (Theoretical maximum value - Theoretical minimum value) × (Upper limit of target interval - Lower limit of target interval) + Lower limit of target interval. This mapping ensures that regardless of the physical unit used for the parameter (radians, degrees, millimeters, percentages, etc.), as long as its theoretical range is known, it can be uniformly scaled to the same standardized numerical range.

[0067] This embodiment linearly maps the second motion state parameter, represented in physical units, to the target numerical range, so that all parameters output in physical units, regardless of their unit system, are mapped to the same standardized numerical space.

[0068] According to some embodiments of this application, when the numerical representation system represents the numerical encoding representation, the step of mapping the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain the second standard format data includes steps S243 to S244.

[0069] In step S243, the lower and upper limits of the original quantization interval defined by the encoding protocol for the joint position of the second type of actuator are obtained from the unified data specification.

[0070] In some embodiments, when the numerical representation system characterizes the numerical encoding representation, the second motion state parameter is output as an encoded integer value. The value itself does not have a direct physical meaning and needs to be mapped to the corresponding physical range according to the encoding protocol defined by the actuator manufacturer. The encoding protocol typically defines the original quantization range, i.e., the minimum and maximum values ​​of the encoded value. For example, the quantization range of a 12-bit encoder is [0, 4095], and the quantization range of a 14-bit encoder is [0, 16383]. This information is obtained from the encoding protocol dictionary pre-stored in the unified data specification. This dictionary is loaded during system initialization and contains encoding protocol parameters for all connected actuator types.

[0071] This embodiment obtains the original quantization range defined by the encoding protocol from the unified data specification, so that the second motion state parameter output in numerical encoding form can establish a correspondence between the encoded value and the physical range.

[0072] In step S244, the second motion state parameters represented by the numerical encoding are linearly mapped to the target numerical range according to the lower and upper limits of the original quantization range.

[0073] In some embodiments, for the second motion state parameter represented by numerical encoding, the following linear mapping formula is used to make it continuous to a standardized interval: Mapped value = (Encoded value - Quantization lower bound) / (Quantization upper bound - Quantization lower bound) × (Target interval upper bound - Target interval lower bound) + Target interval lower bound. This mapping converts discrete integer values ​​into continuous floating-point numbers, while eliminating the differences in numerical distribution caused by different quantization bits between different encoding protocols. The standardized numerical value after mapping no longer retains the resolution differences of the original encoding, and all parameters have the same numerical dimensions and comparison benchmark in the standardized space.

[0074] This embodiment achieves complete elimination of coding heterogeneity by linearly mapping numerical coding parameters to the target numerical range, so that the discretized parameters output by actuators with different coding protocols and different quantization resolutions are all mapped to a continuous standardized numerical space.

[0075] According to some embodiments of this application, a first type of actuator includes one of a robotic arm, a dexterous hand, and a teleoperated device; a second type of actuator includes another of a robotic arm, a dexterous hand, and a teleoperated device; a first motion state parameter includes joint velocity and torque; the physical unit of the joint velocity of the first motion state parameter is expressed in radians per second; the physical unit of the torque of the first motion state parameter is expressed in Newton-meters; a second motion state parameter includes joint velocity and torque; the numerical representation system of the joint velocity of the second motion state parameter is expressed as interval encoding; the numerical representation system of the torque of the second motion state parameter is expressed as Newton-meters.

[0076] According to the pre-configured unified data specification, the first motion state parameter is mapped from physical units to a target numerical range to obtain the first standard format data. According to the pre-configured unified data specification, the second motion state parameter is mapped from the numerical representation system to the same target numerical range as the first motion state parameter after mapping to obtain the second standard format data. This includes step S251: According to the unified data specification, the joint velocities of the first type of actuator and the second type of actuator are uniformly converted into dimensionless relative velocity values, and the torques of the first type of actuator and the second type of actuator are uniformly converted into percentage or normalized torque values.

[0077] In some embodiments, the dimensionless relative velocity value refers to the standardized value obtained by normalizing the joint velocity parameter according to its theoretical maximum velocity and mapping it to the interval [-1, 1] or [0, 1]. The conversion method is: relative velocity = normalized velocity value, which directly reflects the ratio of the current velocity to the actuator's maximum velocity.

[0078] For example, a dimensionless velocity of 0.5 for a robotic arm joint means that its actual speed reaches 50% of its theoretical maximum speed. A dimensionless velocity of 0.3 for a dexterous hand joint means that its actual speed is 30% of its theoretical maximum speed. The two can be directly compared.

[0079] In some embodiments, the normalized torque value is obtained by dividing the actual torque value by the rated torque value, resulting in a standardized value in the interval [0, 1]. If the actual torque value exceeds the rated torque value, the normalized torque value is truncated to 1.0.

[0080] For example, a robotic arm joint has a rated torque of 200 N·m, a currently measured torque of 100 N·m, and a normalized torque value of 0.5. A dexterous hand joint torque code value of 511 corresponds to approximately 2.5 N·m, a rated torque of 5 N·m, and a normalized torque value of 0.5. The two can be directly compared after normalization.

[0081] This embodiment achieves comparability of joint velocities between heterogeneous actuators by uniformly converting the joint velocities of different types of actuators into dimensionless relative velocity values, and uniformly converts torque into normalized values, enabling horizontal comparison of torque load states between actuators of different specifications, and providing a unified quantitative index for load balancing and abnormal overload detection.

[0082] According to some embodiments of this application, step S250 concatenates the mapped first standard format data and second standard format data into a single data structure in a predetermined order, including step S251.

[0083] In step S251, the converted velocity and torque are spliced ​​into a single data structure in a fixed sequence of left arm → right arm → left hand → right hand, which is the same as the joint position.

[0084] In some embodiments, the splicing order is as follows: first, grouping by actuator physical layout (left arm → right arm → left hand → right hand), and then sorting by parameter type within the same actuator (position → velocity → torque). This layout ensures that velocity and torque data have a defined offset relationship with the corresponding position data in the data structure.

[0085] This embodiment ensures that data collected at different times for the same task has a consistent data structure layout by splicing standard format data in a fixed order, which facilitates the construction of batch datasets and model training.

[0086] According to some embodiments of this application, the heterogeneous hardware data acquisition method further includes steps S510 to S530.

[0087] In step S510, the raw state data of the third type of actuator is acquired. The third type of actuator includes an array of tactile sensors. The raw state data includes tactile matrix data with a resolution of 12×6 per finger and the resultant force data of the five fingers. The tactile matrix data is stored in the form of a raw matrix, and the resultant force data is in grams.

[0088] In some embodiments, an array-type tactile sensor refers to a pressure-sensing array deployed on the surface of the fingers of a dexterous hand, with a resolution of 12 rows × 6 columns per finger, totaling 72 sensing units. Each sensing unit outputs the pressure intensity value at that point with 8-bit precision (0 to 255). The raw state data includes the tactile pressure matrix of each finger and the combined force value of the five fingers.

[0089] For example, a dexterous hand is equipped with an array of tactile sensors, with each finger outputting a 12×6 pressure matrix, resulting in five 72-dimensional matrices for all five fingers. It also outputs the combined force of each finger, such as 500 grams for the thumb, 300 grams for the index finger, and 200 grams for the middle finger.

[0090] This embodiment expands the perception dimension of heterogeneous hardware data acquisition systems by acquiring tactile matrix data and resultant force value data, enabling tactile modal data to be incorporated into a unified standardized processing framework.

[0091] In step S520, according to the pre-configured unified data specification, the tactile matrix data is flattened into a 72-dimensional vector in row-major order, and each element in the 72-dimensional vector is converted from the original quantization range of 0-255 into a dimensionless normalized pressure value. The resultant force value data is converted from grams into a dimensionless normalized pressure value.

[0092] In some embodiments, row priority means traversing rows 0 to 11 of the matrix first, and then traversing columns 0 to 5 from left to right within each row, ultimately obtaining a one-dimensional vector of 72 elements. The normalization method is: Normalized pressure value = Original quantized value / 255, mapping the original value in the 0-255 interval to the [0, 1] interval. The resultant force value normalization method is: Normalized resultant force value = Resultant force value (grams) / Sensor maximum range (grams).

[0093] For example, the output value of the sensing unit in the 3rd row and 4th column of a certain finger tactile matrix is ​​128, and the normalized pressure value is 128 / 255≈0.502. The resultant force of this finger is 500 grams, the sensor's maximum range is 1000 grams, and the normalized resultant force value is 0.5.

[0094] This embodiment flattens and normalizes the tactile matrix, placing the tactile modal data and the standardized data of motion state parameters in the same numerical range, thus laying the foundation for multimodal data stitching.

[0095] In step S530, the converted tactile matrix data and the resultant force value data are added as independent fields to a single data structure.

[0096] In some embodiments, a separate field refers to allocating a separate storage area for tactile data (tactile matrix data and resultant force value data) within a single data structure, appended to the motion state parameter data field. The length of the tactile field is determined by the sensor configuration, and the total dimensions of the five-finger tactile data are 5 × (72 + 1) = 365 dimensions.

[0097] For example, the motion state parameter data structure has 66 dimensions, and after adding the five-finger tactile data with 365 dimensions, the expanded single data structure has a total of 66 + 365 = 431 dimensions. Dimensions 1-66 are motion state parameters, and dimensions 67-431 are tactile perception data, which can be directly input into a multimodal neural network.

[0098] This embodiment achieves the fusion storage of motion state data and tactile perception data in the same data structure by adding tactile data to a single data structure, providing a unified data input format for multimodal embodied intelligent models.

[0099] Based on the above-described heterogeneous hardware data acquisition method, this application also provides a heterogeneous hardware data acquisition device. The following will be combined with... Figure 3 The device is described in detail.

[0100] Figure 3 A schematic block diagram of a heterogeneous hardware data acquisition device according to an embodiment of this application is shown.

[0101] like Figure 3 As shown, the heterogeneous hardware data acquisition device 10 includes a first acquisition module 1, a second acquisition module 2, a first mapping module 3, a second mapping module 4, and a splicing module 5.

[0102] The first acquisition module 1 is used to acquire the first motion state parameters of the first type of actuator, and the first motion state parameters are expressed in physical units.

[0103] The second acquisition module 2 is used to acquire the second motion state parameters of the second type of actuator. The second motion state parameters are represented by a numerical representation system.

[0104] The first mapping module 3 is used to map the first motion state parameter from physical units to the target numerical range according to the pre-configured unified data specification, so as to obtain the first standard format data.

[0105] The second mapping module 4 is used to map the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain the second standard format data. The unified data specification defines the mapping relationship from different physical units and numerical representation systems to the unified numerical range.

[0106] The splicing module 5 is used to splice the mapped first standard format data and second standard format data into a single data structure in a predetermined order. Each position in the single data structure can uniquely determine the actuator type, left and right sides, and joint number.

[0107] According to the heterogeneous hardware data acquisition device of this application embodiment, through the coordinated operation of the first acquisition module 1, the second acquisition module 2, the first mapping module 3, the second mapping module 4, and the splicing module 5, the device achieves unified and standardized processing of motion state parameters of different physical units and numerical representation systems of heterogeneous hardware devices, overcoming the problem that data cannot be directly fused and compared between different types of actuators due to inconsistent data formats. By splicing the standardized data into a uniquely traceable single data structure in a predetermined order, it provides structured and standardized multi-actuator fusion input data for downstream machine learning models.

[0108] According to embodiments of this application, any multiple modules among the first acquisition module 1, second acquisition module 2, first mapping module 3, second mapping module 4, and splicing module 5 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the first acquisition module 1, second acquisition module 2, first mapping module 3, second mapping module 4, and splicing module 5 can be at least partially implemented as a hardware circuit, such as a field-programmable gate array, a programmable logic array, a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit, or implemented by any other reasonable means of integrating or packaging the circuit, or implemented by any one of software, hardware, and firmware, or by any appropriate combination of any of these. Alternatively, at least one of the first acquisition module 1, second acquisition module 2, first mapping module 3, second mapping module 4, and splicing module 5 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0109] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing the above-described method according to an embodiment of this application.

[0110] like Figure 4 As shown, an electronic device 900 according to an embodiment of this application includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory 902 or a program loaded from a storage portion 908 into a random access memory 903. The processor 901 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0111] Random access memory 903 stores various programs and data required for the operation of electronic device 900. Processor 901, read-only memory 902, and random access memory 903 are interconnected via bus 904. Processor 901 executes various steps of the method flow according to embodiments of this application by executing programs stored in read-only memory 902 and / or random access memory 903. It should be noted that the programs may also be stored in one or more memories other than read-only memory 902 and random access memory 903. Processor 901 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0112] According to embodiments of this application, the electronic device 900 may further include an input / output interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card, such as a local area network card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0113] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0114] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 902 described above, and / or random access memory 903, and / or one or more memories other than read-only memory 902 and random access memory 903.

[0115] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0116] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0117] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by processor 901, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0118] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A heterogeneous hardware data acquisition method, characterized in that, include: Obtain the first motion state parameter of the first type of actuator, wherein the first motion state parameter is expressed in physical units; Obtain the second motion state parameters of the second type of actuator, wherein the second motion state parameters are represented in a numerical system. According to the pre-configured unified data specification, the first motion state parameter is mapped from the physical unit to the target numerical range to obtain the first standard format data; According to the pre-configured unified data specification, the second motion state parameter is mapped from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, to obtain second standard format data. The unified data specification defines the mapping relationship from different physical units and numerical representation systems to a unified numerical range. The mapped first standard format data and second standard format data are concatenated into a single data structure in a predetermined order, wherein each position in the single data structure can uniquely determine the actuator type, left or right side, and joint number.

2. The heterogeneous hardware data acquisition method according to claim 1, characterized in that, The step of mapping the first motion state parameter from the physical unit to the target numerical range according to the pre-configured unified data specification to obtain the first standard format data includes: Obtain the theoretical minimum and theoretical maximum values ​​of the first motion state parameters from the parameter configuration of the first type of actuator; Based on the theoretical minimum and maximum values ​​obtained from the parameter configuration of the first type of actuator, the first motion state parameter represented by the physical unit is linearly mapped to the target numerical range to obtain the first standard format data.

3. The heterogeneous hardware data acquisition method according to claim 1, characterized in that, The numerical representation system characterizes physical unit representations or numerical code representations. When the numerical representation system represents physical units, the step of mapping the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain the second standard format data, includes: Obtain the theoretical minimum and maximum values ​​of the second motion state parameters from the parameter configuration of the second type of actuator; Based on the theoretical minimum and maximum values ​​obtained from the parameter configuration of the second type of actuator, the motion state parameters represented by the numerical representation system are linearly mapped to the target numerical range to obtain second standard format data. When the numerical representation system represents the numerical encoding representation, the step of mapping the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain the second standard format data, includes: Obtain the lower and upper limits of the original quantization interval defined by the encoding protocol for the joint position of the second type of actuator from the unified data specification; Based on the lower and upper limits of the original quantization interval, the second motion state parameter represented by the numerical code is linearly mapped to the target numerical interval.

4. The heterogeneous hardware data acquisition method according to claim 1, characterized in that, The first type of actuator includes one of a robotic arm, a dexterous hand, and a teleoperated device; the second type of actuator includes another of a robotic arm, a dexterous hand, and a teleoperated device; the first motion state parameters include joint velocity and torque; the physical unit of the joint velocity of the first motion state parameter is expressed in radians per second; the physical unit of the torque of the first motion state parameter is expressed in Newton-meters; the second motion state parameters include joint velocity and torque; the numerical representation system of the joint velocity of the second motion state parameter is interval encoding; the numerical representation system of the torque of the second motion state parameter is Newton-meters. According to a pre-configured unified data specification, the first motion state parameter is mapped from the physical unit to a target numerical range to obtain first standard format data. According to the pre-configured unified data specification, the second motion state parameter is mapped from the numerical representation system to the same target numerical range as the first motion state parameter after mapping to obtain second standard format data, including: According to the unified data specification, the joint velocities of the first type of actuator and the second type of actuator are uniformly converted into dimensionless relative velocity values, and the torques of the first type of actuator and the second type of actuator are uniformly converted into percentage or normalized torque values.

5. The heterogeneous hardware data acquisition method according to claim 4, characterized in that, The step of concatenating the mapped first standard format data and the second standard format data into a single data structure in a predetermined order includes: The converted velocity and torque are assembled into a single data structure in a fixed sequence corresponding to the joint position: left arm → right arm → left hand → right hand.

6. The heterogeneous hardware data acquisition method according to claim 1, characterized in that, Also includes: Acquire raw state data of a third type of actuator, the third type of actuator including an array of tactile sensors, the raw state data including tactile matrix data with a single finger resolution of 12×6 and the combined force value data of the five fingers, the tactile matrix data being stored in raw matrix form, and the combined force value data being in grams; According to the pre-configured unified data specification, the tactile matrix data is flattened into a 72-dimensional vector in row-major order, and each element in the 72-dimensional vector is converted from the original quantization range of 0-255 into a dimensionless normalized pressure value. The resultant force value data is converted from grams into a dimensionless normalized pressure value. The transformed tactile matrix data and the resultant force value data are added as independent fields to the single data structure.

7. A heterogeneous hardware data acquisition device, characterized in that, include: The first acquisition module is used to acquire the first motion state parameters of the first type of actuator, wherein the first motion state parameters are expressed in physical units. The second acquisition module is used to acquire the second motion state parameters of the second type of actuator, wherein the second motion state parameters are represented in a numerical representation system. The first mapping module is used to map the first motion state parameter from the physical unit to the target numerical range according to the pre-configured unified data specification, so as to obtain the first standard format data. The second mapping module is used to map the second motion state parameter from the numerical representation system to the same target numerical range as the first motion state parameter after mapping, according to the pre-configured unified data specification, to obtain second standard format data. The unified data specification defines the mapping relationship from different physical units and numerical representation systems to a unified numerical range. The splicing module is used to splice the mapped first standard format data and the second standard format data into a single data structure in a predetermined order, wherein each position in the single data structure can uniquely determine the actuator type, left and right sides, and joint number.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

Images