Sensor data processing method, device and equipment for intelligent ship

Abstract

The invention provides a sensor data processing method, device and equipment for an intelligent ship. The method comprises the following steps: acquiring first pose data about the intelligent ship acquired by a first sensor and second pose data about the intelligent ship acquired by a second sensor; determining a rotation external parameter relational expression and a translation external parameter relational expression according to the first pose data and the second pose data; determining a rotation external parameter value according to the rotation external parameter relational expression; determining a time deviation between the first sensor and the second sensor; optimizing the rotation external reference value through the time deviation to obtain a target rotation external reference value; determining a target translation external reference value according to the target rotation external reference value and the translation external reference relational expression; and converting the first pose data collected by the first sensor into target data according to the target rotation external reference value and the target translation external reference value. According to the scheme of the invention, camera-IMU joint calibration can be realized, and the pose of the camera is transmitted to the IMU sensor through external parameters to obtain the pose of the camera in an IMU coordinate system.

Description

Technical Field

[0001] The present invention relates to the field of computer information technology processing technology, and in particular to a sensor data processing method, device and equipment for intelligent ships. Background Art

[0002] Visual calibration plays a key role in the implementation of unmanned driving systems and is widely valued by robot manufacturers, vehicle and ship manufacturers, and supply chain manufacturers at all levels. Different types of sensor information in unmanned driving systems are complementary. Fusion of sensor data with different observations can improve the robustness and accuracy of target state perception and estimation algorithms, and improve the scene adaptability of multi-sensor fusion algorithms. High-precision perception based on sensors is the key to the ability of autonomous driving systems to perceive the surrounding environment and the prerequisite for accurate positioning of autonomous driving systems. Multi-sensor joint calibration is a basic requirement for multi-sensor information fusion. High-precision calibration is the basis and prerequisite for the fusion of effective observation information collected by multiple sensors, and can lay a solid foundation for subsequent mapping, positioning, perception and control. Inaccurate calibration results will lead to redundancy and incorrect association of multi-sensor information, affecting the fusion and use of correct observations.

[0003] Camera-IMU (Inertial Measurement Unit) calibration technology is a key technology in the field of multi-sensor fusion, especially in high-precision navigation, attitude estimation, robot vision, unmanned systems and other applications. Accurate calibration of cameras and IMU (Inertial Measurement Unit) is not only the basis for the fusion of image information and inertial data, but also a necessary step to achieve visual inertial navigation and accurately convert the target or scene information in the image into the navigation coordinate system. However, the current calibration method still faces many challenges in terms of accuracy, stability, and adaptability to the application environment. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a sensor data processing method, device and equipment for intelligent ships, which can realize the joint calibration of camera-IMU, transfer the camera posture to the IMU sensor through external parameters, and obtain the precise posture of the camera in the IMU coordinate system.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A sensor data processing method for an intelligent ship, comprising:

[0007] Acquire first pose data about the intelligent ship collected by a first sensor, and second pose data about the intelligent ship collected by a second sensor; the first pose data and the second pose data belong to different coordinate systems;

[0008] Determine a rotational external parameter relational expression and a translational external parameter relational expression for the posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data;

[0009] Determining a rotation extrinsic parameter value according to the rotation extrinsic parameter relationship equation;

[0010] determining a time offset between the first sensor and the second sensor;

[0011] Optimizing the rotation extrinsic parameter value by using the time deviation to obtain a target rotation extrinsic parameter value;

[0012] Determine the target translation extrinsic parameter value according to the target rotation extrinsic parameter value and the translation extrinsic parameter relationship;

[0013] The first pose data collected by the first sensor is converted into target data in a coordinate system of the second pose data according to the target rotation extrinsic parameter value and the target translation extrinsic parameter value.

[0014] Optionally, determining a rotational external parameter relational expression and a translational external parameter relational expression for posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data includes:

[0015] According to the first posture data and the second posture data, a hand-eye calibration relationship for posture conversion is obtained by inter-epoch difference, wherein the hand-eye calibration relationship is XA=BX, where X is an unknown quantity, and A and B are coefficients;

[0016] According to the rotation parameter and the translation parameter, the posture conversion external parameter in the hand-eye calibration relationship is converted to obtain the rotation external parameter relationship and the translation external parameter relationship of the posture conversion of the first sensor and the second sensor.

[0017] Optionally, determining the rotation extrinsic parameter value according to the rotation extrinsic parameter relationship equation includes:

[0018] determining a relative rotational relationship between the poses of the first sensor and the second sensor between two consecutive epochs;

[0019] According to the relative rotation relationship, the rotation external parameter relational expression is processed to obtain a hand-eye calibration relational expression about rotation;

[0020] According to the hand-eye calibration relationship about rotation, epoch iteration is performed to obtain the rotation extrinsic parameter value.

[0021] Optionally, determining a time deviation between the first sensor and the second sensor includes:

[0022] When the first sensor and the second sensor sample data at the same time, determining a difference between a time when the sampled data of the first sensor is received and a time when the sampled data of the second sensor is received;

[0023] The difference is determined as a time offset between the first sensor and the second sensor.

[0024] Optionally, optimizing the rotation extrinsic parameter value by the time deviation to obtain a target rotation extrinsic parameter value includes:

[0025] Determining the position and posture of the first sensor at the sampling time according to the time deviation;

[0026] Determining a relative rotation relationship of the first sensor at two adjacent moments according to the position and posture of the first sensor at the sampling moment;

[0027] The rotational extrinsic parameter value is optimized according to the relative rotational relationship to obtain a target rotational extrinsic parameter value.

[0028] Optionally, determining the target translation extrinsic parameter value according to the relationship between the target rotation extrinsic parameter value and the translation extrinsic parameter value includes:

[0029] Multiplying the translation external parameter relational expression by the target vector to obtain a target equation after eliminating the scale parameter;

[0030] According to the target equation and the target rotation extrinsic parameter value, the target translation extrinsic parameter value is obtained.

[0031] Optionally, converting the first pose data collected by the first sensor into target data in a coordinate system of the second pose data according to the target rotation extrinsic parameter value and the target translation extrinsic parameter value includes:

[0032] According to the target rotation extrinsic parameter value and the target translation extrinsic parameter value, the first pose data in the first three-dimensional coordinate system is converted into pose data in the second three-dimensional coordinate system; the first three-dimensional coordinate system is the coordinate system to which the first pose data belongs, and the second three-dimensional coordinate system is the coordinate system to which the second pose data belongs.

[0033] The present invention also provides a sensor data processing device for an intelligent ship, comprising:

[0034] An acquisition module, used to acquire first posture data about the intelligent ship collected by a first sensor, and second posture data about the intelligent ship collected by a second sensor; the first posture data and the second posture data belong to different coordinate systems;

[0035] A processing module is used to determine a rotation external parameter relationship and a translation external parameter relationship of a first sensor and a second sensor posture conversion according to the first posture data and the second posture data; determine a rotation external parameter value according to the rotation external parameter relationship; determine a time deviation between the first sensor and the second sensor; optimize the rotation external parameter value by the time deviation to obtain a target rotation external parameter value; determine a target translation external parameter value according to the target rotation external parameter value and the translation external parameter relationship; and convert the first posture data collected by the first sensor into target data in the coordinate system of the second posture data according to the target rotation external parameter value and the target translation external parameter value.

[0036] The present invention also provides a computing device, comprising: a processor and a memory storing a computer program, wherein when the computer program is run by the processor, the method as described above is executed.

[0037] The present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method.

[0038] The above solution of the present invention includes at least the following beneficial effects:

[0039] The above scheme of the present invention obtains the first pose data about the intelligent ship collected by the first sensor and the second pose data about the intelligent ship collected by the second sensor; the first pose data and the second pose data belong to different coordinate systems; the rotation external parameter relationship and the translation external parameter relationship of the first sensor and the second sensor pose conversion are determined according to the first pose data and the second pose data; the rotation external parameter value is determined according to the rotation external parameter relationship; the time deviation between the first sensor and the second sensor is determined; the rotation external parameter value is optimized by the time deviation to obtain the target rotation external parameter value; the target translation external parameter value is determined according to the target rotation external parameter value and the translation external parameter relationship; according to the target rotation external parameter value and the target translation external parameter value, the first pose data collected by the first sensor is converted into the target data in the coordinate system of the second pose data. The joint calibration of the camera and the IMU can be realized, and the camera pose is transmitted to the IMU sensor through the external parameter to obtain the precise pose of the camera in the IMU coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a sensor data processing method for an intelligent ship according to an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of camera-IMU extrinsic calibration according to an embodiment of the present invention;

[0042] Figure 3is a flow chart of a camera-IMU extrinsic parameter calibration method according to an embodiment of the present invention;

[0043] Figure 4 This is an example of a camera capturing ship motion in an embodiment of the present invention;

[0044] Figure 5 This is an example of an IMU collecting hull motion in an embodiment of the present invention;

[0045] Figure 6 is a schematic diagram of camera-IMU time deviation according to an embodiment of the present invention;

[0046] Figure 7 Schematic diagram of system deviation between camera attitude and IMU attitude on the time axis in an embodiment of the present invention;

[0047] Figure 8 It is a structural diagram of a sensor data processing device for an intelligent ship according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0049] like Figure 1 As shown, an embodiment of the present invention provides a sensor data processing method for an intelligent ship, comprising:

[0050] Step 11, obtaining first posture data about the intelligent ship collected by the first sensor, and second posture data about the intelligent ship collected by the second sensor; the first posture data and the second posture data belong to different coordinate systems;

[0051] Here, the first sensor refers to a ship-mounted fixed visible light camera, and the second sensor refers to an IMU (inertial measurement unit); the first posture data belongs to a camera coordinate system, and the second posture data belongs to an IMU coordinate system;

[0052] Step 12, determining a rotational external parameter relational expression and a translational external parameter relational expression for the posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data;

[0053] Step 13, determining the rotation extrinsic parameter value according to the rotation extrinsic parameter relationship equation;

[0054] Step 14, determining a time deviation between the first sensor and the second sensor;

[0055] Step 15: Optimize the external rotation parameter value through the time deviation to obtain the target external rotation parameter value;

[0056] Step 16: Determine the target external translation parameter value according to the target external rotation parameter value and the external translation parameter relationship formula;

[0057] Step 17: Convert the first pose data collected by the first sensor into target data in the coordinate system of the second pose data according to the target external rotation parameter value and the target external translation parameter value.

[0058] In this embodiment, as Figure 2 shown, the first pose data collected by the camera is calibrated by the external parameters to realize the conversion from the camera pose to the IMU pose. Among them is the external parameter for the camera-to-IMU conversion, I represents the IMU coordinate system, C represents the camera coordinate system, and calibrating the camera is to determine the value of. Among them Among them is the external rotation parameter described in Step 15, representing the rotation of the three coordinate axes; is the external translation parameter described in Step 16, representing the translation of the coordinate system origin. Calibrating the camera is to determine the target external rotation parameter value and the target external translation parameter value.

[0059] In this embodiment, the visible light camera obtains the pixel observation value of the environment or the target, and the sensor observation registration is to match the pixel observation value of the target of interest. Through the synchronous observation of significant features such as points, lines, and planes (i.e., features of interest) in the environment, the visible light camera obtains the pixel observation value of the feature of interest, and performs conversion by transmitting the central pose, so as to obtain the accurate pose of the sensor in other coordinate systems. The pose is the position and attitude of the observed target.

[0060] In an alternative embodiment of the present invention, Step 12 may include:

[0061] Step 121: According to the first pose data and the second pose data, obtain the hand-eye calibration relationship formula for pose conversion through epoch difference. The hand-eye calibration relationship formula is XA = BX, where X is the unknown quantity, and A and B are coefficients;

[0062] Step 122: Convert the external pose parameter in the hand-eye calibration relationship formula according to the rotation parameter and the translation parameter to obtain the external rotation parameter relationship formula and the external translation parameter relationship formula for the pose conversion between the first sensor and the second sensor.

[0063] In this embodiment, as Figure 3As shown, in order to determine the target rotation extrinsic parameter value and the target translation extrinsic parameter value, three main steps are included, namely, camera-IMU pose relationship construction, camera-IMU time delay and relative rotation solution, and camera-IMU relative translation solution. During the navigation process of the intelligent ship, scene data is collected, feature points are extracted, and the camera pose is calculated in real time. Step 12 in this embodiment is the first step of the three main steps - camera-IMU pose relationship construction.

[0064] To build the camera-IMU pose relationship, we first need to calculate the poses of the camera and IMU respectively. The camera pose is calculated using the open source visual odometry (VO) algorithm and is expressed in the world coordinate system (World, W system) as The IMU pose is calculated using the GNSS / INS combination method, that is, the GNSS RTK position result is combined with the IMU and expressed in the navigation coordinate system (Navigation, N system) as The definitions of the W and N systems are different. In addition, due to the existence of camera-IMU external parameters, the camera trajectory and IMU trajectory do not overlap, such as Figure 4 and Figure 5 Although the camera and IMU trajectories do not overlap, the shapes of their trajectories are similar, but they differ in translation by one scale.

[0065] According to the calculated pose, the camera-IMU pose relationship is constructed. For the conversion from N to W, the camera pose is and IMU pose The transformation relationship can be written as:

[0066]

[0067] For two consecutive moments i and j, the pose relationship between the camera and IMU can be written as follows:

[0068]

[0069]

[0070] in is the position of the camera in the world system at time i, is the position and posture of the IMU in the navigation system at time i, is the position of the camera in the world system at time j, is the position and posture of the IMU in the navigation system at time j.

[0071] Inverting both sides of the first equation of formula (2) yields:

[0072]

[0073] Multiplying the left and right sides of the second equation of formula (3) with formula (2) yields:

[0074]

[0075] in, is the relative position of the camera at time j relative to time i, is the relative pose of the IMU at time j relative to that at time i.

[0076] Eliminate the unknown quantity by the above method Finally, we get the hand-eye calibration relationship of the form XA=BX. Writing the posture T as a combination of rotation R and translation t, we can get:

[0077]

[0078] Among them, s represents the monocular vision scale, represents the rotation matrix of the camera in the world system at time i, Represents the translation vector of the IMU in the world system at time i, represents the rotation matrix of the camera in the world system at time j, represents the rotation matrix of the IMU in the world system at time j, represents the rotation matrix of the camera in the navigation system at time i, represents the translation vector of the IMU in the navigation system at time i, represents the rotation matrix of the camera in the navigation system at time j, Represents the translation vector of the IMU in the navigation system at time j.

[0079] Expand formula (5) and extract the corresponding relationship between rotation and translation respectively, and we get:

[0080]

[0081]

[0082] in, represents the inverse of the rotation matrix of the camera in the world system at time i, Represents the rotation matrix of the IMU relative to the camera, Represents the inverse of the rotation matrix of the camera in the navigation system at time i.

[0083] Formula (6) is the rotation external parameter relational expression in step 122, and formula (7) is the translation external parameter relational expression in step 122. Solve the rotation external parameter according to formula (6) and formula (7): Translate external reference

[0084] In an optional embodiment of the present invention, step 13 may include:

[0085] Step 131, determining a relative rotation relationship between the postures of the first sensor and the second sensor between two consecutive epochs;

[0086] Step 132, processing the rotation external parameter relational expression according to the relative rotation relationship to obtain a hand-eye calibration relational expression for rotation;

[0087] Step 133, performing epoch iteration according to the hand-eye calibration relationship about rotation to obtain rotation extrinsic parameter values.

[0088] In this embodiment, due to the rotation parameter Not affected by translation parameters The latter will be affected by the former, so first determine Re-confirm Also because is coupled with the camera-IMU time delay, so The determination is divided into two steps. First, determine The initial value is then optimized together with the time delay to obtain the target rotation external parameter value. Based on

[0089] In step 13, according to formula (6), let The two represent the relative rotation of the camera and IMU between consecutive epochs i and j. Moving to the left we get:

[0090]

[0091] Formula (8) is the hand-eye calibration relationship for rotation. is the rotation matrix of the camera at time j relative to time i, is the rotation matrix of the IMU at time j relative to time i. The rotation matrix R belongs to the special orthogonal group SO(3) on the manifold and is not easy to solve directly. Write the rotation matrix R in the form of quaternion q, and we can get:

[0092]

[0093]

[0094] in, They are rotation matrices The corresponding quaternion form.

[0095]

[0096] Among them, I 3 represents the 3×3 unit matrix, q w represents the real part of the quaternion q, q xyz A 3×1 vector representing the imaginary part of the quaternion q, Represents vector q xyz Constructed antisymmetric matrix.

[0097] According to formula (9), the equations can be constructed at K moments as follows:

[0098]

[0099] where w k represents the weight of the kth observation equation, which is calculated as where exp(·) represents the exponential function, F R is the scaling factor. Quaternion Solve via weighted least squares:

[0100]

[0101] During the solution, each time an observation of an epoch is added, the equation group (11) is expanded and solved again:

[0102]

[0103] The camera-IMU relative rotation is accurately solved by iteration between epochs. Convert to rotation matrix because Coupling with the camera-IMU time delay, so It is not accurate enough and can only be used as an initial value for the next step of optimization that takes time delay into consideration.

[0104] In an optional embodiment of the present invention, step 14 may include:

[0105] Step 141, when the first sensor and the second sensor sample data at the same time, determining the difference between the time when the sampled data of the first sensor is received and the time when the sampled data of the second sensor is received;

[0106] Step 142: determine the difference as a time offset between the first sensor and the second sensor.

[0107] In this embodiment, Figure 6 As shown in the figure, due to clock inconsistency, transmission delay, sensor response speed, etc., there may be time delays in the camera and IMU timestamps. Assuming that the camera and IMU start sampling at time t, the timestamp is generated after the data is received. for:

[0108]

[0109] in, are the time delays of the camera and IMU respectively. Since the exact value at time t is difficult to obtain, It is also impossible to obtain, but what can be calculated is The difference t d :

[0110]

[0111] like Figure 7 As shown in FIG. 1 , when there is a time deviation, the rotation estimation curves of the two are different. Since the rotation references of the two are different, the vertical axis in the figure is the relative rotation between the two frames.

[0112] In an optional embodiment of the present invention, step 15 may include:

[0113] Step 151, determining the position and posture of the first sensor at the sampling time according to the time deviation;

[0114] Step 152, determining a relative rotation relationship of the first sensor at two adjacent moments according to the position and posture of the first sensor at the sampling moment;

[0115] Step 153: Optimize the rotation extrinsic parameter value according to the relative rotation relationship to obtain a target rotation extrinsic parameter value.

[0116] In this embodiment, for the time deviation, it is necessary to interpolate the camera pose at the sampling time according to the camera pose at the generation time. k ,t k+1 The poses are T k , T k+1 , assuming that the camera moves at a constant speed in this short period of time, the angular velocity and acceleration are:

[0117]

[0118] Among them, ω k , Angular velocity and scale-free linear velocity, respectively. are the rotation matrices of the camera in the world coordinate system at time k and k+1 respectively. is the translation vector of the camera in the world coordinate system at time k+1, and the logarithmic mapping Log(·) converts the rotation matrix into a rotation vector.

[0119] For the time deviation t d , according to ω k , Calculate tk +t d Position at the moment:

[0120]

[0121] The rotation relationship between two consecutive frames k, k+1 is as follows:

[0122]

[0123] in, It is the IMU attitude output by GNSS / INS integrated navigation. is the camera pose output by VO, Represents the rotation matrix of the IMU relative to the camera. Constructing the relative rotation from formula (7), the following relationship can be obtained:

[0124]

[0125] The estimation of external parameter rotation and time deviation is as follows:

[0126]

[0127] During the solution process, each time an observation of an epoch is added, the formula (20) is expanded and solved again. The target rotation extrinsic parameter value is obtained by iteration between epochs:

[0128] In an optional embodiment of the present invention, step 16 may include:

[0129] Step 161, multiplying the translation external parameter relational expression by the target vector to obtain a target equation after eliminating the scale parameter;

[0130] Step 162, obtaining a target translation extrinsic parameter value according to the target equation and the target rotation extrinsic parameter value.

[0131] In this embodiment, since the monocular camera cannot provide accurate scale information, there will be a scale difference between the translation given by VO and the translation given by GNSS / INS combined navigation. After the VO and GNSS / INS combination obtain two sets of poses respectively, the translation parts of the two are associated according to formula (7) as follows:

[0132]

[0133] The above formula includes the scale s and the translation The two unknown parameters make the two parameters have a certain degree of coupling. However, the scale is affected by the error and has drift, and is not a stable parameter in a long time series. If the scale and translation are estimated at the same time, it will affect the translation. Therefore, in this embodiment, the cross-multiplication of the two sides of the formula is get:

[0134]

[0135] The left side of the equation is the 0 vector, and the right side is rearranged to get:

[0136]

[0137] Right now:

[0138]

[0139] The formula is in the form of Ax-B=0. N key frame sequences can form N-1 equation groups, which can be calculated by least squares as follows:

[0140]

[0141] During the solution, the relative rotation between camera and IMU is solved in each iteration Then, based on the historical observations, a new translation observation equation is added for iterative solution to obtain The exact solution of .

[0142] In an optional embodiment of the present invention, step 17 may include:

[0143] Step 171, according to the target rotation extrinsic parameter value and the target translation extrinsic parameter value, convert the first pose data in the first three-dimensional coordinate system into the pose data in the second three-dimensional coordinate system; the first three-dimensional coordinate system is the coordinate system to which the first pose data belongs, and the second three-dimensional coordinate system is the coordinate system to which the second pose data belongs.

[0144] like Figure 2 As shown, Rotate the external parameter value according to the obtained target and target translation extrinsic parameter value The precise pose of the first pose data in the coordinate system of the second pose data can be obtained.

[0145] The above-mentioned embodiment of the present invention adopts the GNSS / INS combined algorithm to obtain the precise posture data of the IMU in real time, and uses the monocular visual mileage calculation method to obtain the posture of the camera. The relative posture between epochs is obtained by inter-epoch difference for both, and the two sets of relative postures are associated according to the robot hand-eye calibration method to construct a posture relationship equation containing the camera-IMU external parameters. Then, the camera-IMU relative rotation and translation are solved step by step, and the camera-IMU relative rotation relationship is extracted from the posture relationship equation constructed in the previous step. At the same time, the camera-IMU time delay is considered, and the camera-IMU time delay and relative rotation are solved simultaneously by weighted least squares; finally, the camera-IMU relative translation relationship is extracted, and a vector cross multiplication method is used to innovatively eliminate the monocular scale parameter, and the camera-IMU translation is solved by weighted least squares. GNSS assistance is introduced, and the posture of the IMU is calculated by the GNSS / INS combined navigation system, which suppresses the error drift in long-term use and improves the calibration accuracy. The time delay and relative rotation of the camera-IMU are solved by synchronous optimization, making the calibration result more accurate and able to meet more practical application needs.

[0146] The above-mentioned embodiments of the present invention aim at the problems of ambiguity of observed targets, large visual ranging errors, and large vibration errors of sensors when the ship moves, which occur in the process of visible light fusion detection of marine targets. Based on the unmanned ship sea surface experimental scene and considering the data characteristics of the ship-borne fixed visible light camera, in-depth research is conducted on the observation errors of ship-borne visible light cameras, joint calibration strategy design, engineering optimization and other technologies, and ship-borne IMU visual calibration software is developed. At the same time, the software meets the requirements of ship-borne calculations. After actual measurement, the angular errors of the XYZ axes after the IMU and the camera are matched are 0.084°, 0.051°, and 0.02°, respectively, which are all less than 0.1°, and the effect is good.

[0147] like Figure 8 As shown, an embodiment of the present invention further provides a sensor data processing device 80 for an intelligent ship, comprising:

[0148] An acquisition module 81 is used to acquire first posture data about the intelligent ship collected by a first sensor, and second posture data about the intelligent ship collected by a second sensor; the first posture data and the second posture data belong to different coordinate systems;

[0149] The processing module 82 is used to determine the rotation external parameter relationship and the translation external parameter relationship of the posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data; determine the rotation external parameter value according to the rotation external parameter relationship; determine the time deviation between the first sensor and the second sensor; optimize the rotation external parameter value by the time deviation to obtain the target rotation external parameter value; determine the target translation external parameter value according to the target rotation external parameter value and the translation external parameter relationship; according to the target rotation external parameter value and the target translation external parameter value, convert the first posture data collected by the first sensor into the target data in the coordinate system of the second posture data.

[0150] Optionally, determining a rotational external parameter relational expression and a translational external parameter relational expression for posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data includes:

[0151] According to the first posture data and the second posture data, a hand-eye calibration relationship for posture conversion is obtained by inter-epoch difference, wherein the hand-eye calibration relationship is XA=BX, where X is an unknown quantity, and A and B are coefficients;

[0152] According to the rotation parameter and the translation parameter, the posture conversion external parameter in the hand-eye calibration relationship is converted to obtain the rotation external parameter relationship and the translation external parameter relationship of the posture conversion of the first sensor and the second sensor.

[0153] Optionally, determining the rotation extrinsic parameter value according to the rotation extrinsic parameter relationship equation includes:

[0154] determining a relative rotational relationship between the poses of the first sensor and the second sensor between two consecutive epochs;

[0155] According to the relative rotation relationship, the rotation external parameter relational expression is processed to obtain a hand-eye calibration relational expression about rotation;

[0156] According to the hand-eye calibration relationship about rotation, epoch iteration is performed to obtain the rotation extrinsic parameter value.

[0157] Optionally, determining a time deviation between the first sensor and the second sensor includes:

[0158] When the first sensor and the second sensor sample data at the same time, determining a difference between a time when the sampled data of the first sensor is received and a time when the sampled data of the second sensor is received;

[0159] The difference is determined as a time offset between the first sensor and the second sensor.

[0160] Optionally, optimizing the rotation extrinsic parameter value by the time deviation to obtain a target rotation extrinsic parameter value includes:

[0161] Determining the position and posture of the first sensor at the sampling time according to the time deviation;

[0162] Determining a relative rotation relationship of the first sensor at two adjacent moments according to the position and posture of the first sensor at the sampling moment;

[0163] The rotational extrinsic parameter value is optimized according to the relative rotational relationship to obtain a target rotational extrinsic parameter value.

[0164] Optionally, determining the target translation extrinsic parameter value according to the relationship between the target rotation extrinsic parameter value and the translation extrinsic parameter value includes:

[0165] Multiplying the translation external parameter relational expression by the target vector to obtain a target equation after eliminating the scale parameter;

[0166] According to the target equation and the target rotation extrinsic parameter value, the target translation extrinsic parameter value is obtained.

[0167] Optionally, converting the first pose data collected by the first sensor into target data in a coordinate system of the second pose data according to the target rotation extrinsic parameter value and the target translation extrinsic parameter value includes:

[0168] According to the target rotation extrinsic parameter value and the target translation extrinsic parameter value, the first pose data in the first three-dimensional coordinate system is converted into pose data in the second three-dimensional coordinate system; the first three-dimensional coordinate system is the coordinate system to which the first pose data belongs, and the second three-dimensional coordinate system is the coordinate system to which the second pose data belongs.

[0169] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0170] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0171] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0172] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0174] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0177] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0178] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0179] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0180] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A sensor data processing method for an intelligent ship, characterized in that: include: Acquire first position data about the intelligent ship collected by the first sensor, and second position data about the intelligent ship collected by the second sensor; The first posture data and the second posture data belong to different coordinate systems; Determine a rotational external parameter relational expression and a translational external parameter relational expression for the posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data; Determining a rotation extrinsic parameter value according to the rotation extrinsic parameter relationship equation; determining a time offset between the first sensor and the second sensor; Optimizing the rotation extrinsic parameter value by using the time deviation to obtain a target rotation extrinsic parameter value; Determine the target translation extrinsic parameter value according to the target rotation extrinsic parameter value and the translation extrinsic parameter relationship; The first pose data collected by the first sensor is converted into target data in a coordinate system of the second pose data according to the target rotation extrinsic parameter value and the target translation extrinsic parameter value.

2. The sensor data processing method for an intelligent ship according to claim 1, characterized in that: Determining a rotational external parameter relational expression and a translational external parameter relational expression for the posture conversion of the first sensor and the second sensor according to the first posture data and the second posture data includes: According to the first posture data and the second posture data, a hand-eye calibration relationship for posture conversion is obtained by inter-epoch difference, wherein the hand-eye calibration relationship is XA=BX, where X is an unknown quantity, and A and B are coefficients; According to the rotation parameter and the translation parameter, the posture conversion external parameter in the hand-eye calibration relationship is converted to obtain the rotation external parameter relationship and the translation external parameter relationship of the posture conversion of the first sensor and the second sensor.

3. The sensor data processing method for an intelligent ship according to claim 1, characterized in that: Determining the rotation extrinsic parameter value according to the rotation extrinsic parameter relationship formula includes: determining a relative rotational relationship between the poses of the first sensor and the second sensor between two consecutive epochs; According to the relative rotation relationship, the rotation external parameter relational expression is processed to obtain a hand-eye calibration relational expression about rotation; According to the hand-eye calibration relationship about rotation, epoch iteration is performed to obtain the rotation extrinsic parameter value.

4. The sensor data processing method for an intelligent ship according to claim 1, characterized in that: Determining a time offset between the first sensor and the second sensor includes: When the first sensor and the second sensor sample data at the same time, determining a difference between a time when the sampled data of the first sensor is received and a time when the sampled data of the second sensor is received; The difference is determined as a time offset between the first sensor and the second sensor.

5. The sensor data processing method for an intelligent ship according to claim 1, characterized in that: Optimizing the rotation extrinsic parameter value by the time deviation to obtain a target rotation extrinsic parameter value includes: Determining the position and posture of the first sensor at the sampling time according to the time deviation; Determining a relative rotation relationship of the first sensor at two adjacent moments according to the position and posture of the first sensor at the sampling moment; The rotational extrinsic parameter value is optimized according to the relative rotational relationship to obtain a target rotational extrinsic parameter value.

6. The sensor data processing method for an intelligent ship according to claim 1, characterized in that: Determining the target translation extrinsic parameter value according to the target rotation extrinsic parameter value and the translation extrinsic parameter relationship includes: Multiplying the translation external parameter relational expression by the target vector to obtain a target equation after eliminating the scale parameter; According to the target equation and the target rotation extrinsic parameter value, the target translation extrinsic parameter value is obtained.

7. The sensor data processing method for an intelligent ship according to claim 1, characterized in that: According to the target rotation extrinsic parameter value and the target translation extrinsic parameter value, converting the first pose data collected by the first sensor into target data in a coordinate system of the second pose data includes: According to the target rotation extrinsic parameter value and the target translation extrinsic parameter value, the first pose data in the first three-dimensional coordinate system is converted into pose data in the second three-dimensional coordinate system; the first three-dimensional coordinate system is the coordinate system to which the first pose data belongs, and the second three-dimensional coordinate system is the coordinate system to which the second pose data belongs.

8. A sensor data processing device for an intelligent ship, characterized in that: include: An acquisition module, used to acquire first posture data about the intelligent ship collected by the first sensor, and second posture data about the intelligent ship collected by the second sensor; The first posture data and the second posture data belong to different coordinate systems; A processing module is used to determine a rotation external parameter relationship and a translation external parameter relationship of a first sensor and a second sensor posture conversion according to the first posture data and the second posture data; determine a rotation external parameter value according to the rotation external parameter relationship; determine a time deviation between the first sensor and the second sensor; optimize the rotation external parameter value by the time deviation to obtain a target rotation external parameter value; determine a target translation external parameter value according to the target rotation external parameter value and the translation external parameter relationship; and convert the first posture data collected by the first sensor into target data in the coordinate system of the second posture data according to the target rotation external parameter value and the target translation external parameter value.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images