A sensor access configuration and data encapsulation method for intelligent controller

Abstract

The application discloses a sensor access configuration and data encapsulation method for an intelligent controller, constructs a sensor configuration file containing various configuration parameters of sensors and writes the sensor configuration file into the intelligent controller; the sensor accesses the intelligent controller: corresponding sensors described in the constructed sensor configuration file are accessed into the intelligent controller; sensor configuration file analysis and data acquisition: when the sensor accesses the intelligent controller, the intelligent controller acquires original data of the sensor by analyzing the sensor configuration file and registers the sensor; sensor data encapsulation: after the sensor accesses the intelligent controller and the sensor configuration file is analyzed, unified data encapsulation is performed on each type of sensor, wherein the unified data encapsulation comprises conversion of unified data format and preprocessing encapsulation; the encapsulated sensor data is transmitted to an application layer of the intelligent controller. The application realizes rapid response and unified data processing of the intelligent controller of an autonomous unmanned system to multiple types of sensors.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent controller application of autonomous unmanned system, and particularly relates to a sensor access configuration and data encapsulation method for an intelligent controller. BACKGROUND

[0002] Autonomous unmanned systems, such as unmanned aerial vehicles, unmanned vehicles, unmanned ships and the like, need to rely on various sensors (including but not limited to speed, attitude, position, ranging, vision and the like) to obtain environmental information and state information, so as to realize autonomous control and decision-making. The above-mentioned sensors mainly cover the following common interfaces in terms of device interfaces: ① Universal Serial Bus (USB) interface; ② Inter-Integrated Circuit (I2C) interface; ③ Serial Peripheral Interface (SPI) interface; ④ Universal Asynchronous Receiver / Transmitter (UART) interface; and ⑤ Ethernet interface. The same type of sensor may include different interface types at the same time, for example, the hardware interface of a vision sensor may include a USB interface and an Ethernet interface. In terms of sensor data formats, the same type of sensor may also be different. For example, the data formats of the common ranging sensors (laser radars) of autonomous unmanned systems are quite different between single-line radars and multi-line radars. The sensor models used by autonomous unmanned systems in different scenarios (sea, land, air and the like) are different, and the types of data obtained are also different.

[0003] In summary, the current cross-domain heterogeneous unmanned system has the problems of carrying various sensors, different data interfaces and high real-time requirements for data interaction, which poses a challenge to the traditional software and hardware integration method and is difficult to meet the convenient integration requirements of the software and hardware of the intelligent controller of the unmanned system. Designing a sensor access configuration and unified encapsulation method for the intelligent controller is the key to solving this challenge. Based on this, a sensor access configuration and data encapsulation method for the intelligent controller is designed. SUMMARY

[0004] In view of the above technical problems, the application provides a sensor access configuration and data encapsulation method for an intelligent controller.

[0005] The technical solution adopted by the application to solve the technical problems is as follows:

[0006] A sensor access configuration and data encapsulation method for an intelligent controller, the method comprising the following steps:

[0007] S100, constructing a sensor configuration file containing configuration parameters of various types of sensors;

[0008] S200, writing the designed sensor configuration file to the intelligent controller;

[0009] S300, sensor access to intelligent controller: access the corresponding sensor described in the constructed sensor configuration file to the intelligent controller;

[0010] S400, sensor configuration file parsing and data acquisition: when the sensor accesses the intelligent controller, the intelligent controller acquires the original data of the sensor by parsing the sensor configuration file, and registers the sensor;

[0011] S500, sensor data encapsulation: after the sensor accesses the intelligent controller and parses the sensor configuration file, unified data encapsulation is performed on each type of sensor, wherein the unified data encapsulation includes conversion of unified data format and preprocessing encapsulation;

[0012] S600, delivering the encapsulated sensor data to the application layer of the intelligent controller.

[0013] Preferably, the sensor types in S100 include speed sensors, position sensors, attitude sensors, ranging sensors, vision sensors, force / torque sensors and tactile sensors.

[0014] Preferably, the unified general field format of the sensor configuration file in S100 includes physical device, sensor name, transmission configuration and transmission protocol; the format of the sensor configuration file is yaml format or xml format.

[0015] Preferably, S100 includes:

[0016] S110, dividing 7 types of typical sensors of autonomous unmanned system according to hardware interface, and configuring the hardware interface; wherein the hardware interface includes USB interface, I2C interface, SPI interface, UART interface and Ethernet interface;

[0017] S120, configuring sensor protocol according to the divided hardware interface type, wherein the parameters of protocol configuration include software type used by the protocol, dynamic library path, and specific configuration parameters of the protocol.

[0018] Preferably, S110 is specifically:

[0019] The sensors transmitting data through the USB interface include speed sensors, position sensors, attitude sensors, ranging sensors, vision sensors, force / torque sensors, and tactile sensors; the sensors transmitting data through the I2C interface include attitude sensors and force / torque sensors; the sensors transmitting data through the SPI interface include attitude sensors and force / torque sensors; the sensors transmitting data through the UART interface include speed sensors, attitude sensors, and force / torque sensors; and the sensors transmitting data through the Ethernet interface include position sensors, ranging sensors, and vision sensors.

[0020] Preferably, S120 specifically includes:

[0021] If the sensor accessing the intelligent controller uses the USB interface, when the sensor is not a vision sensor, the USB device is automatically configured and starts to work normally after being connected to the controller without additional manual configuration; when the sensor is a vision sensor but not a camera device with depth data, additional parameters need to be manually configured, and the parameters configured when accessing include the width and height of the image frame, the data frame format, the device serial number, and the working frequency; when the sensor is a vision sensor and a camera device with depth data, the depth frame parameter should be additionally configured for the access setting;

[0022] If the sensor accessing the intelligent controller uses the UART interface, the parameters configured for the sensor transmission include the device port number, the serial communication stop bit, the serial communication data bit, the serial communication parity bit, the serial communication timeout duration, and the minimum serial communication data bit;

[0023] If the sensor accessing the intelligent controller uses the Ethernet interface, the parameters configured for the sensor transmission include the device IP address and the connection timeout time;

[0024] If the sensor accessing the intelligent controller uses the I2C interface, the parameters configured for the sensor transmission include the IIC mode, the IIC timing duty cycle, the IIC device address, the IIC ACK response signal, the IIC communication rate, and the IIC slave address bit number;

[0025] If the sensor accessing the intelligent controller uses the SPI interface, the parameters configured for the sensor transmission include the SPI mode, the SPI data bit length, the SPI idle phase, the SPI idle polarity, and the SPI data bit leading.

[0026] Preferably, the unified data format conversion in S500 refers to the process of converting the original data format of each type of sensor, such as speed, attitude, position, vision, ranging, touch, and force / torque, into a unified data format; the unified data format includes a basic data format and a measurement data format corresponding to the sensor, wherein the basic data format includes data type, acquisition frequency, frame number, timestamp, and device name.

[0027] Preferably, the measurement data format corresponding to the sensor is specifically:

[0028] The measurement data format of the speed sensor includes acceleration and angular velocity, wherein the acceleration and angular velocity further include data information of X-axis, Y-axis, and Z-axis;

[0029] The measurement data format of the attitude sensor includes pitch angle, roll angle, yaw angle, and magnetic induction intensity, wherein the magnetic induction intensity includes magnetic induction intensity of X-axis, Y-axis, and Z-axis of the sensor body;

[0030] The measurement data format of the position sensor includes longitude information, latitude information, and height information measured based on the geocentric coordinate system;

[0031] The measurement data format of the vision sensor includes image data and depth data, wherein the image data includes channel ID, channel number, image format type, image width, image height, image size, and memory address for storing the image, and the depth data includes channel number, image format type, frame rate, current frame timestamp, depth data width, depth data height, size of the depth data, and memory address for storing the depth data;

[0032] The measurement data format of the ranging sensor includes one-dimensional data format, two-dimensional data format, and three-dimensional data format, wherein the one-dimensional data format includes distance value based on the sensor body coordinate system; the two-dimensional data format includes minimum angle value of sensor scanning, maximum angle value of sensor scanning, angle increment of each measurement, time increment of each measurement, distance of each measurement, scanning time, minimum range of ranging, maximum range of ranging, and intensity value based on the sensor body coordinate system; and the three-dimensional data format includes height of point cloud data, width of point cloud data, data byte step of a single point, byte step of a row of data, array for storing point cloud, whether there is invalid point, whether arranged in ascending order, and attribute of each point in the point cloud data;

[0033] The measurement data format of the touch sensor includes contact position and contact force size, wherein the contact position includes X-axis coordinate and Y-axis coordinate of the contact point;

[0034] The measurement data format of the force / torque sensor includes contact force size and direction, wherein the contact force direction includes X-axis coordinate and Y-axis coordinate.

[0035] Preferably, the data preprocessing encapsulation of the sensor in S500 refers to providing a preprocessing encapsulation interface for the data after the protocol analysis of each type of sensor data after the device accesses the intelligent controller to complete the acquisition and analysis of the original sensor data. The data format after the preprocessing encapsulation interface conforms to the agreement of the unified data format of the sensor.

[0036] Preferably, the data preprocessing encapsulation operation of the corresponding sensor in S500 is specifically:

[0037] The speed sensor data preprocessing encapsulation operation includes filtering of the original data, analysis and denoising of the data stream with interstitial abnormal values, smoothing of the speed data, data interpolation compensation of missing data points, and speed sensor calibration.

[0038] The attitude sensor data preprocessing encapsulation operation includes angular velocity calculation, linear acceleration calculation, fusion and complement of built-in gyroscopes, accelerometers, and magnetometers data.

[0039] The position sensor data preprocessing encapsulation operation includes noise removal of position data, filtering and smoothing of original position data, data correction of interstitial error points, interpolation compensation of missing points, and coordinate system conversion of position data.

[0040] The visual sensor data preprocessing encapsulation operation includes image data format conversion, image data de-distortion, RGB image graying, RGB image binarization, image data edge segmentation, image Gaussian blur, and image erosion.

[0041] The ranging sensor data preprocessing encapsulation operation includes point cloud data boundary point removal, point cloud data range filtering, point cloud data invalid point deletion, point cloud data intensity threshold filtering, point cloud coordinate conversion, overall point cloud data rotation and translation, point cloud data voxel filtering, and scan data filtering.

[0042] The tactile sensor preprocessing encapsulation operation includes noise removal of voltage data, filtering and smoothing of original voltage data, data correction of interstitial error points, interpolation compensation of missing points, calibration of the tactile sensor, force conversion of voltage data, and coordinate system conversion of voltage data.

[0043] The force / torque sensor preprocessing encapsulation operation includes noise removal of voltage data, filtering and smoothing of original voltage data, data correction of interstitial error points, interpolation compensation of missing points, calibration of the force / torque sensor, force conversion of voltage data, and direction conversion of the contact force of voltage data.

[0044] The aforementioned sensor access configuration and data encapsulation method for intelligent controllers involves several steps. Sensor access configuration refers to configuring sensor parameters, interfaces, and communication methods to achieve rapid integration between the sensor and the intelligent controller's hardware and software system. This ensures the controller can quickly respond to sensor access and accurately acquire sensor measurement data, meeting the real-time response and dynamic management requirements of the autonomous unmanned system's intelligent controller system. Sensor data encapsulation involves format conversion and preprocessing of the raw data acquired by the sensor before providing it to the application layer. This method enables the autonomous unmanned system's intelligent controller to quickly respond to multiple types of sensor access and perform unified data processing, reducing the development difficulty of the application layer. Attached Figure Description

[0045] Figure 1 This is an overall flowchart of a sensor access configuration and data encapsulation method for intelligent controllers according to an embodiment of the present invention;

[0046] Figure 2 This is the overall content of the sensor configuration file in one embodiment of the present invention;

[0047] Figure 3 This is a unified data format and preprocessing encapsulation framework for seven typical sensors in one embodiment of the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0049] In one embodiment, such as Figure 1 As shown, a sensor access configuration and data encapsulation method for intelligent controllers includes the following steps:

[0050] S100: Build a sensor configuration file that includes various configuration parameters of the sensor.

[0051] In one embodiment, the sensor types in S100 include speed sensors, position sensors, attitude sensors, ranging sensors, vision sensors, force / torque sensors, and tactile sensors.

[0052] In one embodiment, the unified general field format of the sensor configuration file in S100 includes physical device, sensor name, transmission configuration, and transmission protocol; the sensor configuration file is in YAML or XML format.

[0053] Specifically, the typical sensors carried by the autonomous unmanned system mainly include seven types of sensors, i.e., speed, position, attitude, ranging, vision, force / torque, and touch. Before the sensors are connected to the intelligent controller, the sensor configuration file is written to the controller to realize the rapid response of the intelligent controller to the sensor connection. The unified general field format of the sensor configuration file is as follows: (1) physical device, (2) sensor name, (3) transmission configuration, (4) transmission protocol, as shown in Table 1. The style of the sensor configuration file can adopt the yaml format or the xml format, and the overall content of the sensor configuration file is as shown in Table 1. Figure 2

[0054] Table 1 Sensor connection configuration table

[0055] Symbol Definition phydevice Physical device name Sensor name transport Transport configuration class_name Software class used by transport lib_path Dynamic library path param Transport specific configuration parameters protocol Transport protocol class_name Software class used by protocol lib_path Dynamic library path param Protocol specific configuration parameters

[0056] In one embodiment, S100 includes:

[0057] S110, dividing the seven types of typical sensors of the autonomous unmanned system according to hardware interfaces, and configuring the hardware interfaces; wherein the hardware interfaces include USB interface, I2C interface, SPI interface, UART interface and Ethernet interface.

[0058] S120, configuring the sensor protocol according to the divided hardware interface type, wherein the parameters of the protocol configuration include the software class used by the protocol, the dynamic library path, and the specific configuration parameters of the protocol.

[0059] Specifically, each sensor needs to configure the related parameters of transmission, i.e., the transport field, when connected to the intelligent controller, including 1) the software class used for transmission, 2) the dynamic library path, and 3) the specific configuration parameters of transmission, as shown in Table 2. Different sensors use different interfaces, so when configuring the related parameters of transmission, the specific interface needs to be combined for configuration, as shown in Table 2. Figure 2

[0060] Table 2 Sensor transmission configuration

[0061]

[0062] In one embodiment, S110 is specifically:

[0063] ​​The sensors transmitting data through the USB interface include speed sensors, position sensors, attitude sensors, ranging sensors, vision sensors, force / torque sensors and tactile sensors; the sensors transmitting data through the I2C interface include attitude sensors and force / torque sensors; the sensors transmitting data through the SPI interface include attitude sensors and force / torque sensors; the sensors transmitting data through the UART interface include speed sensors, attitude sensors and force / torque sensors; and the sensors transmitting data through the Ethernet interface include position sensors, ranging sensors and vision sensors.

[0064] In one embodiment, S120 specifically comprises:

[0065] If the sensor accessing the intelligent controller uses the USB interface, when the sensor is not a vision sensor, the USB device will be automatically configured and start normal work after being connected to the controller without additional manual configuration; when the sensor is a vision sensor but not a camera device with depth data, additional parameters need to be manually configured, and the parameters configured when accessing include the width and height of the image frame, the data frame format, the device serial number and the working frequency, as shown in Table 3; when the sensor is a vision sensor and a camera device with depth data, the depth frame parameter should be additionally configured, as shown in Table 4.

[0066] Table 3 USB interface configuration

[0067]

[0068] Table 4 Depth frame parameter

[0069]

[0070]

[0071] If the sensor accessing the intelligent controller uses the UART interface, the parameters configured by the sensor transmission include the device port number, the serial communication stop bit, the serial communication data bit, the serial communication parity bit, the serial communication timeout length and the serial communication minimum data bit, as shown in Table 5.

[0072] Table 5 UART interface configuration

[0073]

[0074] If the sensor accessing the intelligent controller uses the Ethernet interface, the parameters configured by the sensor transmission include the device IP address and the connection timeout time, as shown in Table 6.

[0075] Table 6 Ethernet interface configuration

[0076]

[0077] If the sensor connected to the intelligent controller uses an I2C interface, the sensor transmission configuration parameters include IIC mode, IIC timing duty cycle, IIC device address, IIC ACK response signal, IIC communication rate, and IIC slave address bit number, as shown in Table 7.

[0078] Table 7 IIC interface configuration

[0079]

[0080]

[0081] If the sensor connected to the intelligent controller uses an SPI interface, the sensor transmission configuration parameters include SPI mode, SPI data bit length, SPI idle phase, SPI idle polarity, and SPI data bit lead.

[0082] Table 8 SPI interface configuration

[0083]

[0084] S200, write the designed sensor configuration file to the intelligent controller.

[0085] S300, sensor access intelligent controller: access the corresponding sensor described in the constructed sensor configuration file to the intelligent controller.

[0086] S400, sensor configuration file parsing and data acquisition: when the sensor is connected to the intelligent controller, the intelligent controller acquires the original data of the sensor by parsing the sensor configuration file, and registers the sensor.

[0087] Specifically, the original data of the sensor includes the model, name, manufacturer, driver file path, and sensor data storage information of the sensor.

[0088] S500, sensor data encapsulation: after the sensor is connected to the intelligent controller and the sensor configuration file is parsed, each type of sensor is uniformly encapsulated, wherein the uniform data encapsulation includes uniform data format conversion and preprocessing encapsulation.

[0089] In one embodiment, the uniform data format conversion in S500 refers to the process of converting the original data format of each type of sensor such as speed, attitude, position, vision, ranging, touch, and force / torque into a uniform data format; the uniform data format includes a basic data format and a measurement data format corresponding to the sensor, wherein the basic data format includes data type, acquisition frequency, frame number, timestamp, and device name.

[0090] Specifically, in the conversion process of the unified data format, firstly, the common basic data format for describing the sensor measurement data of each type of sensor is designed and defined, then the unified data format of each type of sensor is obtained by adding the corresponding sensor measurement data format on the basis data format of the sensor, and finally the original data format of the sensor is converted into the defined unified data format. The basic data format of the sensor includes data type, collection frequency, frame number, time stamp and device name, wherein the data type refers to the type or format of the data, which is used to describe the nature and structure of the data; the collection frequency refers to the frequency of data collection, i.e. the number of data points collected per second, which indicates the speed and accuracy of data collection; the frame number refers to the number of data frames, which is used to identify each frame in the data, and through the frame number, the data can be sorted and organized during data transmission or storage; the time stamp refers to the time mark when the data is collected, and the time stamp is used to record the time point of data collection, which can be used for sorting and time-related analysis applications; and the device name refers to the name or identifier of the device for data collection. The device name is used to identify the type, location, uniqueness and other information of the data collection device, facilitating the management of the data. The basic data format is shown in Table 9.

[0091] Table 9 Sensor basic data format

[0092]

[0093] As shown in Figure 3 , in one embodiment, the measurement data format corresponding to the sensor is specifically:

[0094] The measurement data format of the speed sensor includes acceleration and angular velocity, wherein the acceleration and angular velocity each contain data information of X-axis, Y-axis and Z-axis.

[0095] Specifically, the unified data format of the speed sensor contains acceleration and angular velocity information on the basis data format, wherein the acceleration refers to the rate of change of the speed of an object per unit time, which is used to describe the acceleration or deceleration process of the object; and the angular velocity refers to the rate of change of the angle of an object per unit time, which is used to describe the rotation speed and rotation direction of the object. The acceleration and angular velocity information each contain data information of X-axis, Y-axis and Z-axis. The unified data format of the speed sensor is shown in Table 10.

[0096] Table 10 Measurement data format of speed sensor

[0097]

[0098] The measurement data format of the attitude sensor includes pitch angle, roll angle, yaw angle and magnetic induction intensity, wherein the magnetic induction intensity contains the magnetic induction intensity of the X-axis, Y-axis and Z-axis of the aircraft body.

[0099] Specifically, the unified data format of the attitude sensor contains the pitch angle, the roll angle, the yaw angle and the magnetic induction intensity on the basis data format, wherein the pitch angle refers to the angle between the object rotating around the horizontal axis and the reference plane (usually the horizontal plane), which describes the degree of forward and backward inclination of the object; the roll angle refers to the angle between the object rotating around the longitudinal axis and the reference plane, which describes the degree of left and right inclination of the object; the yaw angle refers to the angle between the object rotating around the vertical axis and the reference direction, which describes the change of the rotation direction of the object; the magnetic induction intensity refers to the magnetic field intensity around the object, which is used to deduce the direction and position of the object by measuring the geomagnetic field, and the magnetic induction intensity contains the magnetic induction intensity of the X-axis, Y-axis and Z-axis of the machine body. The unified data format of the attitude sensor is shown in Table 11.

[0100] Table 11 Measurement data format of attitude sensor

[0101]

[0102] The measurement data format of the position sensor includes the longitude information, the latitude information and the height information measured based on the geocentric coordinate system.

[0103] Specifically, the unified data format of the position sensor contains the longitude information, the latitude information and the height information measured based on the geocentric coordinate system on the basis data format. The longitude information is used to determine the position of a location in the east-west direction; the latitude is used to determine the position of a location in the south-north direction; and the height information is used to describe the altitude or vertical position of a location relative to the earth's surface. The unified data format of the attitude sensor is shown in Table 12.

[0104] Table 12 Measurement data format of position sensor

[0105]

[0106] The measurement data format of the visual sensor includes image data and depth data, wherein the image data contains the channel ID, the channel number, the image format type, the image width, the image height, the image size, the memory address storing the image, the depth data contains the channel number, the image format type, the frame rate, the current frame timestamp, the depth data width, the depth data height, the size of the depth data and the memory address storing the depth data.

[0107] The unified data format of the visual sensor can be divided into image data and depth data on the basis data format. The image data includes channel ID, channel number, image format type, image width, image height, image size, and memory address storing the image. The channel ID is used to identify the unique identifier or index of different channels; the channel number is used to represent the digital number of different channels, usually starting from 0; the image format type refers to the encoding format or storage format of the image data; the image width refers to the number of pixels or pixel columns of the image data in the horizontal direction; the image height refers to the number of pixels or pixel rows of the image data in the vertical direction; the image size refers to the storage space size occupied by the image data; and the memory address storing the image refers to the storage location of the image data in the computer memory. It is a unique address value for accessing and operating the image data. The unified data format of the visual sensor image data is shown in Table 13.

[0108] Table 13: Measurement data format of visual sensor image

[0109]

[0110] The depth image includes channel number, image format type, frame rate, current frame timestamp, depth data width, depth data height, depth data size, and memory address storing the depth data. The unified data format of the visual sensor depth data is shown in Table 14.

[0111] Table 14: Measurement data format of visual sensor depth image

[0112]

[0113] The measurement data format of the ranging sensor includes one-dimensional data format, two-dimensional data format, and three-dimensional data format. The one-dimensional data format includes distance value based on the sensor body coordinate system; the two-dimensional data format includes minimum angle value scanned by the sensor based on the sensor body coordinate system, maximum angle value scanned by the sensor, angle increment of each measurement, time increment of each measurement, distance of each measurement, scanning time, minimum range of ranging, maximum range of ranging, and intensity value; and the three-dimensional data format includes height of point cloud data, width of point cloud data, data byte step of a single point, byte step of a row of data, array storing the point cloud, whether there is invalid point, whether arranged in ascending order, and attribute of each point in the point cloud data.

[0114] Specifically, the unified data format of the ranging sensor can be divided into one-dimensional data format, two-dimensional data format, and three-dimensional data format on the basis data format. The one-dimensional data format includes distance value based on the sensor body coordinate system. The one-dimensional measurement data format of the ranging sensor is shown in Table 15.

[0115] Table 15 Range sensor one-dimensional measurement data format

[0116]

[0117] The two-dimensional data format contains the minimum angle value of sensor scanning, the maximum angle value of sensor scanning, the angle increment of each measurement, the time increment of each measurement, the distance of each measurement, the scanning time, the minimum range of ranging, the maximum range of ranging, and the intensity value, all based on the sensor body coordinate system. The minimum angle value of sensor scanning refers to the minimum angle range that the sensor can cover during scanning, indicating the minimum resolution or accuracy of sensor scanning. The maximum angle value of sensor scanning refers to the maximum angle range that the sensor can cover during scanning, indicating the maximum coverage range of sensor scanning. The angle increment of each measurement refers to the angle value changed by the sensor during each measurement during scanning. The time increment of each measurement refers to the time interval between each measurement during scanning. The distance of each measurement refers to the distance value obtained by the sensor during each measurement during scanning. The scanning time refers to the time required for the sensor to complete a complete scan. The minimum range of ranging refers to the shortest distance that the sensor can accurately measure. The maximum range of ranging refers to the farthest distance that the sensor can accurately measure. The intensity value refers to the intensity or intensity level of the reflected signal obtained by the sensor during measurement, indicating the reflection characteristics or signal intensity of the target object. The two-dimensional measurement data format of the ranging sensor is shown in Table 16.

[0118] Table 16 Range sensor two-dimensional measurement data format

[0119]

[0120] The three-dimensional data format contains the height of point cloud data, the width of point cloud data, the data byte step of a single point, the byte step of a row of data, the array for storing point cloud, whether there are invalid points, whether arranged in ascending order, and the attributes of each point in the point cloud data, all based on the sensor body coordinate system. The height of point cloud data refers to the dimension or number of rows in the vertical direction of the point cloud data. The width of point cloud data refers to the dimension or number of columns in the horizontal direction of the point cloud data. The data byte step of a single point refers to the size of the data representing a point in storage. The byte step of a row of data refers to the size of the data representing a row of point cloud data in storage. The array for storing point cloud refers to the data structure used to store point cloud data, usually a two-dimensional array. The presence of invalid points refers to the possibility of some invalid or unreliable points in the point cloud data. Arranged in ascending order refers to whether the points in the point cloud data are arranged in a certain order. The attributes of each point in the point cloud data refer to additional information or characteristics contained by each point. The three-dimensional measurement data format of the ranging sensor is shown in Table 17.

[0121] Table 17 Range sensor 3D measurement data format

[0122]

[0123] The measurement data format of the tactile sensor includes a contact position and a contact force size, wherein the contact position includes X-axis and Y-axis coordinates of the contact point.

[0124] Specifically, the unified data format of the tactile sensor includes a contact position and a contact force size on the basis data format, wherein the contact position refers to the position of the contact point between the object and the sensor perceived by the tactile sensor; the contact force size refers to the size of the contact force exerted by the object on the sensor perceived by the tactile sensor. The contact position includes X-axis and Y-axis coordinates of the contact point. The unified data format of the tactile sensor is shown in Table 18.

[0125] Table 18 Measurement data format of tactile sensor

[0126]

[0127] The measurement data format of the force / torque sensor includes a contact force size and a contact force direction, wherein the contact force direction includes X-axis and Y-axis coordinates.

[0128] Specifically, the unified data format of the force / torque sensor includes a contact force size and a contact force direction on the basis data format, wherein the contact force direction includes X-axis and Y-axis coordinates. The unified data format of the force / torque sensor is shown in Table 19.

[0129] Table 19 Measurement data format of force / torque sensor

[0130]

[0131] In one embodiment, the data preprocessing encapsulation of the sensor in S500 refers to providing a preprocessing encapsulation interface for the data parsed from each type of sensor data protocol after the device accesses the intelligent controller to complete the acquisition and analysis of the raw sensor data, and the data format after passing through the preprocessing encapsulation interface conforms to the agreement of the unified data format of the sensor.

[0132] Specifically, the data preprocessing encapsulation refers to providing a unified preprocessing encapsulation interface for the data parsed from each type of sensor, so as to perform corresponding preprocessing operations according to the user's requirements. Specifically, after obtaining the parsed data of the sensor, a data preprocessing interface is provided for each type of sensor, and the user only needs to pass the required preprocessing method, and the interface will perform corresponding processing, and output the processing result in a unified data format.

[0133] As Figure 3As shown, in one embodiment, the data preprocessing and packaging operation of the corresponding sensor in S500 is as follows:

[0134] The speed sensor data preprocessing and packaging operation includes filtering of raw data, noise removal of data stream with mixed outliers, smoothing of speed data, data interpolation compensation of missing data points, and speed sensor calibration;

[0135] The attitude sensor data preprocessing and packaging operation includes angular velocity calculation, linear acceleration calculation, fusion of built-in gyroscope, accelerometer and magnetometer data;

[0136] The position sensor data preprocessing and packaging operation includes noise removal of position data, filtering and smoothing of raw position data, data correction of mixed error points, interpolation compensation of missing points, and coordinate system conversion of position data;

[0137] The visual sensor data preprocessing and packaging operation includes image data format conversion, image data rectification, RGB image grayscale, RGB image binarization, image data edge segmentation, image Gaussian blur, and image erosion;

[0138] The ranging sensor data preprocessing and packaging operation includes point cloud data boundary point removal (noise removal), point cloud data range filtering, point cloud data invalid point deletion, point cloud data intensity threshold filtering, point cloud coordinate conversion, overall point cloud data rotation and translation, point cloud data voxel filtering, and scan data filtering;

[0139] The tactile sensor preprocessing and packaging operation includes noise removal of voltage data, filtering and smoothing of raw voltage data, data correction of mixed error points, interpolation compensation of missing points, calibration of tactile sensor, force conversion of voltage data, and coordinate system conversion of voltage data;

[0140] The force / torque sensor preprocessing and packaging operation includes noise removal of voltage data, filtering and smoothing of raw voltage data, data correction of mixed error points, interpolation compensation of missing points, calibration of force / torque sensor, force conversion of voltage data, and direction conversion of voltage data.

[0141] S600, the packaged sensor data is transmitted to the intelligent controller application layer.

[0142] Specifically, after obtaining the data in a unified data format, a message bus is used as a communication middleware for data transmission. Sensor data can be transmitted through a publish-subscribe mode or a request-response mode.

[0143] The sensor access configuration and data encapsulation method for the intelligent controller is configured by configuring parameters, interfaces and communication modes of the sensor, so that the sensor can be quickly integrated with the software and hardware system of the intelligent controller, the controller can quickly respond to the access of the sensor, and the measurement data of the sensor can be correctly acquired to meet the real-time response and dynamic management of the sensor access of the intelligent controller system of the autonomous unmanned system.

[0144] The sensor access configuration and data encapsulation method for the intelligent controller is configured by configuring parameters, interfaces and communication modes of the sensor, so that the sensor can be quickly integrated with the software and hardware system of the intelligent controller, the controller can quickly respond to the access of the sensor, and the measurement data of the sensor can be correctly acquired to meet the real-time response and dynamic management of the sensor access of the intelligent controller system of the autonomous unmanned system. The principle and implementation mode of the present application are described by using specific examples in the present application. The above description of the embodiments is only used to help understand the core idea of the present application. It should be pointed out that, for those skilled in the art, without departing from the principle of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A sensor access configuration and data encapsulation method for intelligent controllers, characterized in that, The method includes the following steps: S100: Construct a sensor configuration file containing various configuration parameters of the sensor; S100 includes: S110. Divide the seven typical sensors of the autonomous unmanned system according to the hardware interface and configure the hardware interface; the hardware interface includes USB interface, I2C interface, SPI interface, UART interface and Ethernet interface. S120. Configure the sensor protocol according to the divided hardware interface type. The protocol configuration parameters include the software class used by the protocol, the dynamic library path, and the specific protocol configuration parameters. If the sensor connected to the smart controller uses a USB interface, and the sensor is not a vision sensor, the USB device will automatically configure and start working normally when connected to the controller, without the need for additional manual configuration. When the sensor is a vision sensor but not a camera device with depth data, additional parameters need to be manually configured. The parameters configured during connection include the width and height of the image frame, the data frame format, the device serial number, and the operating frequency. When the sensor is a vision sensor and a camera device with depth data, the connection settings should additionally configure the depth frame parameters. S200: Write the designed sensor configuration file to the intelligent controller; S300, Sensor Access to Intelligent Controller: Connect the corresponding sensor described in the constructed sensor configuration file to the intelligent controller; S400 Sensor Configuration File Parsing and Data Acquisition: When a sensor is connected to the intelligent controller, the intelligent controller parses the sensor configuration file, acquires the sensor's raw data, and registers the sensor. S500 Sensor Data Encapsulation: After the sensor is connected to the intelligent controller and the sensor configuration file is parsed, the data of each type of sensor is uniformly encapsulated. The uniform data encapsulation includes the conversion and preprocessing of the uniform data format. S600 transmits the packaged sensor data to the application layer of the intelligent controller.

2. The method according to claim 1, characterized in that, The sensor types in the S100 include speed sensors, position sensors, attitude sensors, distance sensors, vision sensors, force / torque sensors, and tactile sensors.

3. The method according to claim 2, characterized in that, The unified and common field format of the sensor configuration file in S100 includes physical device, sensor name, transmission configuration and transmission protocol; the sensor configuration file is in YAML or XML format.

4. The method according to claim 3, characterized in that, S110 specifically refers to: Sensors that transmit data via USB interface include speed sensors, position sensors, attitude sensors, distance sensors, vision sensors, force / torque sensors, and tactile sensors; sensors that transmit data via I2C interface include attitude sensors and force / torque sensors. Sensors that transmit data via the SPI interface include attitude sensors and force / torque sensors; Sensors that transmit data via the UART interface include speed sensors, attitude sensors, and force / torque sensors; Sensors that transmit data via Ethernet interfaces include position sensors, distance sensors, and vision sensors.

5. The method according to claim 4, characterized in that, S120 also includes: If the sensor connected to the smart controller uses a UART interface, the parameters for sensor transmission configuration include the device port number, serial communication stop bit, serial communication data bits, serial communication parity bit, serial communication timeout duration, and serial communication minimum data bits. If the sensor connected to the smart controller uses an Ethernet interface, the parameters for sensor transmission configuration include the device IP address and connection timeout. If the sensor connected to the smart controller uses an I2C interface, the parameters for sensor transmission configuration include IIC mode, IIC timing duty cycle, IIC device address, IIC ACK response signal, IIC communication rate, and IIC slave address bit width. If the sensor connected to the smart controller uses an SPI interface, the parameters for sensor transmission configuration include SPI mode, SPI data bit length, SPI idle phase, SPI idle polarity, and SPI data bit first.

6. The method according to claim 5, characterized in that, In S500, the unified data format conversion refers to the process of converting the raw data format of each type of sensor, such as velocity, attitude, position, vision, ranging, tactile, and force / torque, into a unified data format. The unified data format includes the basic data format and the measurement data format of the corresponding sensor. The basic data format includes data type, acquisition frequency, frame number, timestamp, and device name.

7. The method according to claim 6, characterized in that, The specific measurement data format for the corresponding sensor is as follows: The measurement data format of the velocity sensor includes acceleration and angular velocity, where acceleration and angular velocity respectively contain data information of the X-axis, Y-axis and Z-axis; The measurement data format of the attitude sensor includes pitch angle, roll angle, yaw angle, and magnetic flux density, among which the magnetic flux density includes the magnetic flux density in the three directions of the machine body: X-axis, Y-axis, and Z-axis. The measurement data format of the position sensor includes longitude, latitude, and altitude information measured based on the geocentric coordinate system; The measurement data format of a vision sensor includes image data and depth data. The image data includes channel ID, channel number, image format type, image width, image height, image size, and memory address where the image is stored. The depth data includes channel number, image format type, frame rate, current frame timestamp, depth data width, depth data height, depth data size, and memory address where the depth data is stored. The measurement data formats of ranging sensors include one-dimensional, two-dimensional, and three-dimensional data formats. The one-dimensional data format contains the distance value based on the sensor's body coordinate system. The two-dimensional data format contains the minimum angle value of the sensor scan, the maximum angle value of the sensor scan, the angle increment for each measurement, the time increment for each measurement, the distance for each measurement, the scan time, the minimum range of distance measurement, the maximum range of distance measurement, and the intensity value, all based on the sensor's body coordinate system. The three-dimensional data format contains the height of the point cloud data, the width of the point cloud data, the data byte step size for a single point, the byte step size for a row of data, the array storing the point cloud, whether there are invalid points, whether the data is arranged in ascending order, and the attributes of each point in the point cloud data. The measurement data format of the tactile sensor includes the contact position and the magnitude of the contact force, where the contact position includes the X-axis coordinate and Y-axis coordinate of the contact point; The measurement data format of the force / torque sensor includes the magnitude and direction of the contact force, where the direction of the contact force includes the coordinates of the X and Y axes.

8. The method according to claim 7, characterized in that, In S500, sensor data preprocessing and encapsulation refers to providing a preprocessing and encapsulation interface for the data parsed by each type of sensor data protocol after the device is connected to the intelligent controller to complete the acquisition and parsing of the sensor's raw data. The data format after passing through the preprocessing and encapsulation interface conforms to the unified data format convention of the sensor.

9. The method according to claim 8, characterized in that, The data preprocessing and encapsulation operations for the corresponding sensors in S500 are as follows: The speed sensor data preprocessing and encapsulation operations include filtering the raw data, parsing and denoising the data stream containing outliers, smoothing the speed data, interpolating and compensating for missing data points, and calibrating the speed sensor. The attitude sensor data preprocessing and packaging operations include angular velocity calculation, linear acceleration calculation, and fusion and complementation of data from the built-in gyroscope, accelerometer, and magnetometer. The preprocessing and packaging operations for position sensor data include noise removal from position data, filtering and smoothing of raw position data, data correction for miscellaneous error points, interpolation compensation for missing points, and coordinate system transformation of position data. Visual sensor data preprocessing and packaging operations include image data format conversion, image data distortion correction, RGB image grayscale conversion, RGB image binarization, image data edge segmentation, image Gaussian blurring, and image erosion. The preprocessing and encapsulation operations for ranging sensor data include point cloud data boundary point removal, point cloud data range filtering, point cloud data invalid point deletion, point cloud data intensity threshold filtering, point cloud coordinate transformation, rotation and translation of the overall point cloud data, voxel filtering of point cloud data, and scan data filtering. The preprocessing and packaging operations for tactile sensors include noise removal of voltage data, filtering and smoothing of raw voltage data, data correction of miscellaneous error points, interpolation compensation of missing points, calibration of the tactile sensor, force conversion of voltage data, and coordinate system transformation of voltage data. The preprocessing and packaging operations for force / torque sensors include noise removal of voltage data, filtering and smoothing of raw voltage data, data correction for inclusion error points, interpolation compensation for missing points, calibration of the force / torque sensor, force conversion of voltage data, and direction conversion of contact force in voltage data.

Images