A shifting method for an electrically assisted bicycle and an electrically assisted bicycle

Abstract

The application relates to a gear shifting method of an electrically-assisted bicycle and the electrically-assisted bicycle, the electrically-assisted bicycle comprising a visual sensor and an inertial sensor, the method comprising: obtaining target point cloud data based on an environment image acquired by the visual sensor; converting the target point cloud data to a gravity alignment coordinate system based on pose data acquired by the inertial sensor and an external parameter calibration relationship between the visual sensor and the inertial sensor; performing surface fitting on the target point cloud data in the gravity alignment coordinate system to obtain a fitted surface; determining an uphill turning point, a slope angle and a turning point distance based on the fitted surface; determining a control strategy based on the slope angle, the turning point distance and a current riding condition of the electrically-assisted bicycle; and controlling the electrically-assisted bicycle to automatically shift gears based on the control strategy within a preset distance before the uphill turning point is reached, so that gear switching is completed in advance before the vehicle reaches the uphill turning point, and safe and smooth uphill transition is realized.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method for shifting gears in an electric-assisted bicycle and the electric-assisted bicycle itself. Background Technology

[0002] The automatic gearing function of electric-assist bicycles can automatically switch gears according to the riding status, which significantly improves the smoothness and comfort of riding, reduces the rider's attention to gear operation during riding, and allows them to focus more on road conditions and riding itself.

[0003] Existing automatic transmission technology mainly relies on real-time collected vehicle motion parameters such as cadence and pedaling torque as the basis for gear shifting decisions. This type of solution can basically meet the needs of use on flat roads, but it has obvious shortcomings when facing terrain changes (especially uphill and downhill scenarios): due to the lack of perception of terrain information, e-bikes cannot respond to changes in slope in time, resulting in a mismatch between gear shifting and actual riding load, causing uneven electric assist output and affecting the riding experience.

[0004] Therefore, how to enable electric-assisted bicycles to anticipate the terrain ahead and actively respond to changes in slope to improve the riding experience is a technical problem that needs to be solved. Summary of the Invention

[0005] Based on this, a gear shifting method for an electric-assisted bicycle and an electric-assisted bicycle are provided to solve the technical problem that electric-assisted bicycles cannot actively respond to changes in slope.

[0006] In a first aspect, embodiments of this application provide a method for shifting gears in an electric-assisted bicycle. The electric-assisted bicycle includes a vision sensor and an inertial sensor. The vision sensor is used to acquire environmental images of the riding direction in real time, and the inertial sensor is used to acquire the pose data of the electric-assisted bicycle in real time. The method includes:

[0007] Based on the environmental image, target point cloud data is obtained;

[0008] Based on the pose data and the extrinsic parameter calibration relationship between the visual sensor and the inertial sensor, the target point cloud data is converted to a gravity-aligned coordinate system;

[0009] Under the gravity-aligned coordinate system, the target point cloud data is fitted with a surface to obtain a fitted surface;

[0010] Based on the fitted surface, the uphill inflection point of the fitted surface is determined; and based on the uphill inflection point, the slope angle between the uphill surface and the horizontal plane, and the distance between the uphill inflection point and the inflection point of the current electric-assisted bicycle are determined.

[0011] Based on the slope angle, the distance to the turning point, and the current riding conditions of the electric-assisted bicycle, a control strategy is determined; and within a preset distance before reaching the uphill turning point, the electric-assisted bicycle is automatically shifted based on the control strategy.

[0012] In one embodiment, obtaining target point cloud data based on the environmental image includes:

[0013] Depth estimation is performed on each pixel in the environmental image to obtain a depth image corresponding to the environmental image;

[0014] Based on the pixel coordinates of each pixel in the environmental image, the depth value of each pixel in the depth image, and the camera intrinsic parameter matrix, each pixel is transformed from the pixel coordinate system to the camera coordinate system to obtain the original point cloud data.

[0015] The original point cloud data is used to extract road regions to obtain target point cloud data.

[0016] In one embodiment, road region extraction is performed on the original point cloud data to obtain target point cloud data, including:

[0017] The environmental image is semantically segmented based on a semantic segmentation network to obtain a binary segmentation mask for the road region.

[0018] The original point cloud data is extracted based on the binary segmentation mask to obtain the target point cloud data;

[0019] Determine whether the target point cloud data meets the preset accuracy;

[0020] If not, then a region of interest is selected from the depth image based on spatial geometric prior rules, and the original point cloud data is extracted based on the region of interest to obtain the target point cloud data.

[0021] In one embodiment, converting the target point cloud data to a gravity-aligned coordinate system based on the pose data and the extrinsic parameter calibration relationship between the visual sensor and the inertial sensor includes:

[0022] Based on the pose data and the extrinsic parameter calibration relationship between the visual sensor and the inertial sensor, a rotation matrix is ​​constructed from the camera coordinate system to the gravity-aligned coordinate system;

[0023] Based on the rotation matrix, the target point cloud data is transformed from the camera coordinate system to the gravity-aligned coordinate system, wherein the Y-axis of the gravity-aligned coordinate system points along the horizontal plane towards the riding direction, the Z-axis points directly upward along the opposite direction of gravity, and the X-axis points to the right of the riding direction along the horizontal plane.

[0024] In one embodiment, surface fitting is performed on the target point cloud data to obtain a fitted surface, including:

[0025] The target point cloud data is divided into several vertical slices at equal intervals along the Y-axis direction of the gravity-aligned coordinate system.

[0026] A plane fitting method is used to fit each of the longitudinal slices into a local sub-plane;

[0027] By splicing the local sub-planes along the riding direction, a fitted surface is obtained.

[0028] In one embodiment, determining the upslope inflection point of the fitted surface based on the fitted surface includes:

[0029] Extract the normal vectors of each of the local sub-planes;

[0030] Based on the normal vectors of each adjacent local sub-plane, the deflection angle of each adjacent local sub-plane is determined;

[0031] If the deflection angle is greater than a preset angle threshold, the slope inflection point is determined based on the intersection line of the adjacent local sub-planes; and based on the normal vector of the adjacent local sub-planes, it is determined whether the slope inflection point is an uphill inflection point.

[0032] In one embodiment, determining the slope angle between the uphill surface and the horizontal plane based on the uphill inflection point includes:

[0033] Based on the normal vector of the local sub-plane after the uphill inflection point, a continuous local sub-plane is determined;

[0034] The upslope surface is determined based on the continuous local sub-planes;

[0035] The slope angle between the upslope surface and the horizontal plane is determined based on the average normal vector of the continuous local sub-planes.

[0036] In one embodiment, determining the upslope inflection point of the fitted surface based on the fitted surface includes:

[0037] Differentiating the fitted surface yields the slope variation curve along the cycling direction;

[0038] Based on the slope change curve, the upslope inflection point of the fitted surface is obtained.

[0039] In one embodiment, the control strategy is determined based on the slope angle, the distance to the turning point, and the current riding condition of the electric-assisted bicycle, including:

[0040] Based on the slope angle, determine the target gear.

[0041] Based on the target gear, the distance to the turning point, and the current riding conditions of the electric-assisted bicycle, a control strategy is determined, wherein the riding conditions include the current gear, current speed, and current cadence.

[0042] Secondly, embodiments of this application also provide an electric-assisted bicycle, including a vision sensor, an inertial sensor, and a controller, wherein the controller is connected to the vision sensor and the inertial sensor to implement the method described in the first aspect above.

[0043] The aforementioned gear shifting method and the electric-assist bicycle itself incorporate visual perception into the automatic gear shifting decision-making process. An environmental image of the road ahead is acquired via a camera, and depth estimation is performed to obtain target point cloud data. Combined with pose data from an inertial sensor, the target point cloud data is converted to a gravity-aligned coordinate system and fitted to a surface. Based on the fitted surface, the uphill turning point and slope angle of the terrain ahead are estimated in advance, allowing gear shifting to be completed before the vehicle reaches the uphill turning point. This solution enables gear shifting to be completed in advance under conditions of sufficient vehicle speed and stable transmission system load, avoiding the risk of low-speed, high-load shifting shock and chain damage caused by delayed gear shifting. It also eliminates the problem of riders struggling to start after losing speed before an incline, achieving a safe and smooth uphill transition and significantly improving the riding experience.

[0044] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a flowchart illustrating the gear shifting method of an electric-assisted bicycle in one embodiment;

[0047] Figure 2 This is a schematic diagram of a gravity-aligned coordinate system in one embodiment;

[0048] Figure 3 This is a flowchart illustrating the gear shifting method of an electric-assisted bicycle in another embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0050] While some related technologies utilize inertial measurement units (IMUs) to acquire information about the vehicle's current terrain slope to assist in automatic gear shifting decisions, IMUs are essentially real-time sensing devices. They can only detect the vehicle's current attitude and slope, and cannot predict upcoming terrain changes. This limitation results in a persistent lag in gear shifting relative to terrain changes. For example, when a vehicle is about to enter a steep uphill section, the system may only trigger a downshift after the slope has actually changed, by which time the vehicle speed has often significantly decreased or even approached a stall. Shifting under these conditions can, on the one hand, cause stress on the transmission system due to low-speed, high-load gear changes, increasing the risk of chain breakage; on the other hand, the rider may find it difficult to restart due to an excessively high gear, severely impacting riding safety and continuity.

[0051] Based on this, this application provides a gear shifting method for an electric-assisted bicycle. The electric-assisted bicycle includes an onboard vision sensor and an onboard inertial sensor, which are rigidly connected as a single unit. The relative pose relationship between the two has been predetermined through calibration. During riding, the vision sensor acquires a video stream of the riding direction in real time and then obtains the environmental image of the current frame from the video stream. The inertial sensor includes an accelerometer and a gyroscope, which measures acceleration and angular velocity in real time. By performing attitude calculation on the raw measurement data, the attitude data of the current bicycle body relative to the direction of gravity is obtained.

[0052] Figure 1 This is a flowchart illustrating the gear shifting method for an electric-assisted bicycle provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps:

[0053] Step S201: Obtain target point cloud data based on the environmental image.

[0054] The target point cloud data is the point cloud data of the road ahead of the cyclist. First, a visual sensor is used to acquire the current environmental image, and then operations such as depth estimation, coordinate system transformation, and region of interest extraction are performed on the environmental image to obtain the target point cloud data.

[0055] The visual sensor can be a monocular camera, consisting of only one camera. The monocular camera acquires a single environmental image at the current moment, lacking depth information. Therefore, a deep learning-based monocular depth estimation algorithm (such as UniDepth) or a motion reconstruction method can be used to infer the scene depth from the single image. The motion reconstruction method requires prior information such as the vertical distance from the camera's mounting position to the ground to assist in recovering the true scale. In some embodiments, the visual sensor can be a binocular camera, consisting of two fixed cameras placed side-by-side, simulating human eyes. It acquires a pair of binocular images at the current moment, and calculates depth information using the disparity of the binocular image pair. A binocular stereo matching algorithm (such as the semi-global matching algorithm SGM) can be used for depth estimation.

[0056] Depth values ​​of each pixel in the environmental image are obtained through a depth estimation algorithm, generating a depth map corresponding to the environmental image. Then, based on the camera intrinsic parameter matrix K, each pixel in the depth map is back-projected into a 3D space in the camera coordinate system to generate a 3D point cloud. Finally, the region of interest is extracted from the 3D point cloud to obtain the road target point cloud data.

[0057] Step S202: Based on the pose data and the extrinsic calibration relationship between the visual sensor and the inertial sensor, the target point cloud data is converted to a gravity-aligned coordinate system.

[0058] The inertial sensor is an IMU (Inertial Measurement Unit), which includes an accelerometer and a gyroscope. The accelerometer measures triaxial acceleration in real time, and the gyroscope measures triaxial angular velocity in real time. By performing attitude calculations on the real-time accelerometer and gyroscope measurement data (acceleration and angular velocity) output by the IMU, the current vehicle attitude data (i.e., roll and pitch angles) relative to the direction of gravity can be obtained. Attitude calculations can employ methods such as complementary filtering and extended Kalman filtering (EKF) (e.g., the Mahony complementary filtering algorithm), or visual inertial odometry (VIO) methods (e.g., OpenVINS, OKVIS). This method achieves higher accuracy by fusing camera and IMU data to jointly estimate the pose.

[0059] A gravity-aligned coordinate system is a horizontal coordinate system in which one axis (usually the Z-axis or Y-axis) is strictly parallel to the direction of gravity, and the other two axes are perpendicular to the direction of gravity. This step transforms the target point cloud data from the original coordinate system to the gravity-aligned coordinate system. After this transformation, the Z-axis or Y-axis coordinates of each point in the point cloud directly reflect its true height relative to the horizontal plane, eliminating the influence of the vehicle's tilt and providing a reliable geometric basis for subsequent slope calculations.

[0060] Step S203: Under the gravity alignment coordinate system, perform surface fitting on the target point cloud data to obtain the fitted surface.

[0061] Surface fitting is a technique that approximates discrete point cloud data using a mathematical model to obtain a continuous surface. In this application, the fitted surface can be used to extract road surface information such as slope and undulation. Specifically, a fitted surface can be obtained by directly performing global fitting on all road point cloud data. Preferably, a fitting window can be set to perform segmented fitting on local road point cloud data and then stitch the segments together to obtain the fitted surface.

[0062] Step S204: Based on the fitted surface, determine the uphill inflection point of the fitted surface; and based on the uphill inflection point, determine the slope angle between the uphill surface and the horizontal plane, as well as the distance between the uphill inflection point and the inflection point of the current electric-assisted bicycle.

[0063] During cycling, uphill riding is a core scenario requiring automatic transmissions. When facing an increasing gradient, riders are prone to stalling and chain overload if they fail to downshift in time. In contrast, downhill riding typically involves gliding or light pedaling, resulting in a lower load on the drivetrain. Therefore, this step, based on a fitted surface, identifies uphill turning points and the gradient angle between the uphill surface and the horizontal plane in advance. It then estimates the remaining time to reach these turning points based on the distance and current speed, allowing for pre-shifting based on the gradient angle before reaching the turning point.

[0064] Step S205: Based on the slope angle, the distance to the turning point, and the current riding conditions of the electric-assisted bicycle, determine a control strategy; and within a preset distance before reaching the uphill turning point, control the electric-assisted bicycle to automatically shift gears based on the control strategy.

[0065] The riding conditions of the electric-assist bicycle include the current gear, current speed, and current cadence. This application makes decisions based on the slope angle, the distance to the turning point, and the current riding conditions of the electric-assist bicycle to determine the corresponding control strategy. The control strategy is used to initiate a downshift command in advance when there is sufficient remaining time before reaching the uphill turning point, lowering the riding gear to the target gear to increase the torque output of the motor and pedals. This allows for stronger climbing thrust with a slower wheel speed, avoiding the risk of low-speed, high-load shifting shock and chain damage caused by delayed gear shifting. The target gear can be determined based on the preset mapping relationship between the slope angle and the gear. The riding gears of the electric-assist bicycle are set according to the number of freewheels, such as first, second, third, fourth, and fifth gears. Since gear adjustment is completed before the slope change occurs and while the vehicle is still in a stable riding state, the rider will not feel a significant interruption of power or shock during the entire uphill transition, significantly improving the smoothness and comfort of the riding experience.

[0066] The aforementioned gear shifting method for electric-assist bicycles incorporates visual perception into the automatic gear shifting decision-making process. It acquires environmental images of the road ahead via a camera, performs depth estimation on these images to obtain road target point cloud data, and combines this with pose data acquired by an inertial sensor. The target point cloud data is then converted to a gravity-aligned coordinate system and fitted to a surface. Based on the fitted surface, uphill turning points, slope angles, and distances to these turning points are identified and estimated in advance, allowing gear shifting to be completed before the vehicle reaches the uphill turning point. This approach enables gear shifting to be completed in advance under conditions of sufficient speed and stable transmission system load, avoiding the risks of low-speed, high-load shifting shocks and chain damage caused by delayed gear shifting. It also eliminates the problem of riders struggling to start after losing speed before an incline, achieving a safe and smooth uphill transition and significantly improving the riding experience.

[0067] In one embodiment, obtaining target point cloud data based on the environmental image includes the following steps:

[0068] Step S301: Perform depth estimation on each pixel in the environment image to obtain a depth image corresponding to the environment image.

[0069] If the camera is a monocular camera, a deep learning-based monocular depth estimation algorithm (such as UniDepth) is used for depth estimation; if the camera is a stereo camera, a stereo matching algorithm (such as the semi-global matching algorithm SGM) is used for depth estimation.

[0070] Step S302: Based on the pixel coordinates of each pixel in the environmental image, the depth value of each pixel in the depth image, and the camera intrinsic parameter matrix, each pixel is transformed from the pixel coordinate system to the camera coordinate system to obtain the original point cloud data.

[0071] Based on the camera intrinsic parameter matrix K, each pixel in the depth map is back-projected into a 3D space in the camera coordinate system to generate a 3D point cloud. Let the coordinates of a pixel in the environment image be (u, v), and its corresponding depth value be d. Then the 3D coordinates P of this pixel in the camera coordinate system are... c = (X_c, Y_c, Z_c) is calculated by the following formula:

[0072]

[0073] Where K is the camera intrinsic parameter matrix, which includes the focal length (f_x, f_y) and the principal point coordinates (c_x, c_y). X_c represents the distance of the pixel from the optical center in the horizontal direction, Y_c represents the distance of the pixel from the optical center in the vertical direction, and Z_c represents the distance of the pixel from the optical center along the optical axis.

[0074] Step S303: Extract road regions from the original point cloud data to obtain target point cloud data.

[0075] This step performs Region of Interest (ROI) filtering on the generated raw point cloud to exclude interference points from non-road areas (such as walls, green belts, vehicles, and other obstacles) and extract the target point cloud data for the road area. ROI filtering can be implemented using any of the following methods: (a) In the depth image space, select rectangular ROI pixel regions based on heuristic rules and directly extract the corresponding point cloud subset. This method has low computational cost and is suitable for scenarios with regular road structures. Heuristic rules do not rely on complex mathematical optimization or machine learning models, but instead use intuitive, empirical, and low-computational-cost simple strategies. Essentially, they utilize prior knowledge or common-sense constraints to extract target point cloud data from the depth map. (b) In the original environment image space, use a semantic segmentation network to perform pixel-level segmentation of the rideable area, generate a binary mask, and then select the corresponding point cloud subset from the target point cloud data based on this mask. This method has stronger adaptability to complex road environments. The semantic segmentation network can be built based on the Transformer network architecture. (c) In the 3D point cloud space, directly define a 3D bounding box for point cloud region filtering.

[0076] In one embodiment, the process of extracting road regions from the original point cloud data to obtain target point cloud data includes the following steps: performing semantic segmentation on the environment image based on a semantic segmentation network to obtain a binary segmentation mask for the road region; extracting the original point cloud data based on the binary segmentation mask to obtain target point cloud data; determining whether the target point cloud data meets a preset accuracy; if it does not meet the preset accuracy, discarding the target point cloud data obtained based on semantic segmentation, and directly selecting a region of interest from the depth image using spatial geometric prior rules (i.e., heuristic rules). This region of interest is a narrow rectangular pixel region in the direction of cycling. Then, extracting the original point cloud data based on the region of interest to obtain target point cloud data.

[0077] In this application embodiment, semantic segmentation method is preferred, and the semantic segmentation result is detected as good. If the semantic segmentation result is bad, it automatically rolls back to the image space heuristic rule selection method to ensure the accuracy of target point cloud data extraction.

[0078] In one embodiment, converting the target point cloud data to a gravity-aligned coordinate system based on the pose data and the extrinsic calibration relationship between the visual sensor and the inertial sensor includes the following steps:

[0079] Step S401: Based on the pose data and the extrinsic parameter calibration relationship between the visual sensor and the inertial sensor, construct a rotation matrix from the camera coordinate system to the gravity-aligned coordinate system.

[0080] Step S402: Based on the rotation matrix, the target point cloud data is transformed from the camera coordinate system to the gravity-aligned coordinate system.

[0081] The definition of the gravity-aligned coordinate system is as follows: Figure 2 As shown, the origin of the gravity-aligned coordinate system is located at the optical center of the camera. The Y-axis points along the horizontal plane in the direction of riding (directly forward), the Z-axis points directly upward in the opposite direction of gravity, and the X-axis points to the right of the direction of riding along the horizontal plane. This coordinate system constitutes a standard East-North-Sky (ENU) right-handed coordinate system.

[0082] This application constructs a rotation matrix R from the camera coordinate system to the gravity-aligned coordinate system based on the attitude data obtained from the IMU and the pre-calibrated extrinsic parameter relationship between the camera and the IMU. g Based on rotation matrix R g The coordinates of each point in the point cloud are transformed from the camera coordinate system to the gravity-aligned coordinate system. The coordinate transformation formula is as follows:

[0083]

[0084] in, Represents the three-dimensional coordinates of a pixel in the gravity-aligned coordinate system, in degrees; P represents the rotation matrix from the camera coordinate system to the gravity-aligned coordinate system. c Represents the three-dimensional coordinates of a pixel in the camera coordinate system, in degrees.

[0085] In one embodiment, a piecewise fitting method is used for surface fitting. The process of fitting the target point cloud data to obtain the fitted surface includes the following steps:

[0086] Step S501: Divide the target point cloud data into several vertical slices at equal intervals along the Y-axis direction of the gravity alignment coordinate system.

[0087] First, the target point cloud data is downsampled and filtered: voxel downsampling is performed on the target point cloud in the gravity-aligned coordinate system to reduce the point cloud density and reduce subsequent computation. At the same time, statistical filtering is used to remove outlier noise points and retain the main road structure. Then, longitudinal slicing and subplane fitting are performed: the point cloud is divided into several longitudinal slices at equal intervals along the Y-axis (cycling direction). Each slice corresponds to a continuous local region in the cycling direction. Subplane fitting is then performed within each slice.

[0088] Step S502: Using a plane fitting method, each of the longitudinal slices is fitted into a local sub-plane.

[0089] Within each slice region, a local sub-plane is fitted using a plane fitting method, which can be either the least squares method or the Random Sample Consensus Algorithm (RANSAC).

[0090] Step S503: The local sub-planes are spliced ​​together along the riding direction to obtain the fitted surface.

[0091] In this embodiment, the point cloud is longitudinally sliced ​​along the riding direction, and a local sub-plane is fitted in each slice area. The slope is calculated by extracting the normal vector sequence of the local sub-plane. The three-dimensional terrain analysis is reduced to a one-dimensional sequence mutation detection problem, which significantly improves the robustness of terrain estimation and reduces the computational complexity of real-time processing.

[0092] In one embodiment, determining the upslope inflection point of the fitted surface based on the fitted surface includes the following steps:

[0093] Step S601: Extract the normal vector of each local sub-plane.

[0094] Extract the normal vectors of each local sub-plane, where the normal vector of the i-th local sub-plane is n_i = (n_{ix}, n_{iy}, n_{iz}). This forms an ordered sequence of normal vectors {n_1, n_2, ..., n_N} along the riding direction, where N is the total number of slices.

[0095] Step S602: Determine the deflection angle of each adjacent local sub-plane based on the normal vector of each adjacent local sub-plane.

[0096] Let the normal vector of the i-th slice (local subplane) be n_i, then the deflection angle θ_i of adjacent normal vectors can be calculated by the following formula:

[0097]

[0098] in, Let be the normal vector of the (i+1)th slice.

[0099] Step S603: If the deflection angle is greater than a preset angle threshold, then the slope inflection point is determined based on the intersection line of the adjacent local sub-planes; and based on the normal vector of the adjacent local sub-planes, it is determined whether the slope inflection point is an uphill inflection point.

[0100] The identification of slope inflection points is mainly based on the deflection angle of adjacent normal vectors. When the deflection angle θ_i exceeds the preset threshold θ_th, it is determined that there is a slope abrupt change between the i-th slice and the (i+1)-th slice. Then, the slope inflection point is determined by the intersection line of the i-th slice and the (i+1)-th slice. The coordinate value of the slope inflection point in the Y-axis direction is the distance of the inflection point from the front of the current vehicle position, i.e., the inflection point distance.

[0101] After identifying the slope inflection point, the normal vector of the slice is used to determine whether it is an uphill inflection point. Specifically, the sub-plane normal vector n_i = (n_{ix}, n_{iy}, n_{iz}) fitted to each slice reflects not only the degree of inclination of the local road surface but also the direction of inclination. In the gravity-aligned coordinate system, the Z-axis is strictly aligned with the direction of gravity, so each component of the normal vector has a clear physical meaning: the n_{iz} component reflects the degree of deviation of the road surface from the horizontal plane, while the sign of the n_{iy} component (the component along the riding direction) directly indicates the inclination direction of the local road surface relative to the horizontal plane; a positive value corresponds to uphill, and a negative value corresponds to downhill (or vice versa, depending on the definition of the positive direction of the coordinate axis). Therefore, when the deflection angle θ_i exceeds the threshold θ_th, indicating a slope abrupt change, further combining the sign and trend of the n_{iy} component in the slice normal vector before and after the inflection point allows for a unique determination of whether the abrupt change is from flat road to uphill, flat road to downhill, or a transition between uphill and downhill.

[0102] In one embodiment, determining the slope angle between the uphill surface and the horizontal plane based on the uphill inflection point includes the following: determining a series of local sub-planes based on the normal vectors of the local sub-planes after the uphill inflection point; determining the uphill surface based on the series of local sub-planes; and determining the slope angle between the uphill surface and the horizontal plane based on the average normal vector of the series of local sub-planes.

[0103] In this embodiment, after identifying the uphill inflection point, the continuous slice area following the inflection point is marked as the uphill surface. The continuous slice area exhibits continuity and consistency in the Y-axis direction. Continuity means the slices are adjacent; consistency means the normal vector directions of all slices within the area are essentially the same (the included angles are less than a threshold, preventing the triggering of new inflection points). Based on these continuity and consistency characteristics, the continuous slice area following the uphill inflection point is determined as the uphill surface. Then, the mean direction of the plane normal vector of this continuous slice area is taken as the normal vector n_slope of the uphill surface. The slope angle α of the uphill surface relative to the horizontal plane is calculated by the following formula:

[0104]

[0105] Where, n_slope = ( , These three components correspond to the magnitudes of the normal vector along the X, Y, and Z axes, respectively.

[0106] Here, n_slope has been normalized to a unit vector. The slope angle α and the inflection point distance d_transition obtained from this are provided as outputs to the automatic transmission controller of the electric-assist bicycle for pre-adjusting the gear position.

[0107] Alternatively, the slope angle α can be calculated using the angle between the normal vector and the vertical direction (Z-axis):

[0108]

[0109] In one embodiment, determining the uphill inflection point of the fitted surface based on the fitted surface includes: differentiating the fitted surface to obtain a slope change curve along the cycling direction; and obtaining the uphill inflection point of the fitted surface based on the slope change curve.

[0110] This application embodiment performs overall surface fitting on the road target point cloud, for example, using a B-spline surface algorithm. The derivative of a B-spline surface is continuous, allowing for accurate calculation of slope changes at any location. By differentiating the fitted surface, the slope change curve along the cycling direction is obtained. This slope change curve is analyzed to identify uphill inflection points, and the slope angle α between the uphill surface and the horizontal plane is obtained. This method can obtain a continuous slope distribution, but it has high computational complexity and stricter requirements on point cloud quality and density.

[0111] In one embodiment, determining the control strategy based on the slope angle, the distance to the turning point, and the current riding conditions of the electric-assisted bicycle includes the following steps:

[0112] Step S701: Determine the target gear based on the slope angle.

[0113] The target gear can be determined based on the preset mapping relationship between the slope angle and the gear.

[0114] Step S702: Determine a control strategy based on the target gear, the distance to the turning point, and the current riding conditions of the electric-assisted bicycle.

[0115] The cycling conditions described include the current gear, current speed, and current cadence. First, the remaining time to reach the uphill transition point is estimated based on the distance to the transition point and the current speed. Then, the target gear is determined based on the gradient angle. When there is sufficient remaining time to reach the uphill transition point, a downshift command is initiated in advance based on the control strategy. For example, when the distance to the transition point d_transition is less than a preset advance shift distance threshold d_th, and the gradient angle α is greater than the gradient trigger threshold α_th, a downshift is triggered.

[0116] Furthermore, if the target gear and the current gear jump across multiple gears at once, in order to avoid the impact caused by jumping across multiple gears at once, a step-by-step downshifting strategy can be set. Each time, downshift one gear and wait for the transmission system to stabilize before deciding whether to continue downshifting. The gear is downshifted to the target gear at a preset distance (several meters to more than ten meters) before reaching the uphill turning point, which achieves smooth gear switching and improves riding comfort.

[0117] In some embodiments, structured light or Time-of-Flight (ToF) depth cameras can directly output depth maps without the need for additional depth estimation algorithms. However, such visual sensors typically have a short effective ranging range and their performance degrades in strong outdoor light conditions, limiting their applicability. In other embodiments, LiDAR can be used instead of cameras as 3D sensing devices to directly acquire point cloud data of the road ahead, eliminating the need for image depth estimation. However, LiDAR is significantly more expensive than cameras, and it is also larger, heavier, and less vibration-resistant, which is detrimental to the lightweight design of electric-assist bicycles and their durability in everyday use.

[0118] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0119] Based on the same inventive concept, this application also provides an electric-assisted bicycle, such as... Figure 3 As shown, the electric-assist bicycle includes a vision sensor, an inertial sensor, and a controller. The controller is connected to the vision sensor and the inertial sensor, and is used to implement the method described in any of the above embodiments.

[0120] The controller employs an embedded computing platform with a heterogeneous computing architecture. The CPU handles general computing tasks such as inertial data attitude calculation, coordinate system transformation, point cloud preprocessing, surface fitting, slope calculation, and control strategy generation. For highly parallel and computationally intensive tasks in the visual front-end, such as mono / binocular depth estimation and semantic segmentation, an NPU or GPU accelerates the processing. In one specific embodiment, the computing platform can utilize a SoC chip that integrates a multi-core CPU and a high-performance NPU, meeting real-time requirements while balancing power consumption and cost, thus adapting to the onboard deployment needs of electric-assist bicycles.

[0121] The controller sends the control strategy to the ECU gear control subsystem via a CAN bus in the form of messages. The controller also encapsulates the calculated uphill turning point distance, slope angle, and target gear position, among other slope prediction signals, into CAN bus messages, which are then sent to the ECU gear control subsystem of the electric-assist bicycle via the CAN bus. The ECU gear control subsystem, based on the received calculated messages and control strategy messages, drives the electronic shift actuator to complete automatic gear shifting. Using a CAN bus for message interaction provides high real-time performance, strong anti-interference capabilities, and a mature bus arbitration mechanism, meeting the reliable, low-latency transmission requirements between slope prediction signals and gear shifting control commands in a vehicle environment.

[0122] In a specific embodiment, such as Figure 3 As shown, the visual sensor installed on the electric-assist bicycle is a camera, and the inertial sensor is an IMU sensor. The controller receives the raw video stream acquired in real time by the camera, and the accelerometer and gyroscope data measured in real time by the IMU sensor. The controller performs image preprocessing, road segmentation and depth estimation, and point cloud generation and segmentation operations on the raw video stream in sequence to obtain target point cloud data. The controller performs noise filtering, sensor fusion, and current pose estimation operations on the measurement data from the accelerometer and gyroscope in sequence to obtain pose data. The controller combines the pose data and converts the target point cloud data to a gravity-aligned coordinate system, slices it in the gravity-aligned coordinate system, and performs surface fitting based on the sliced ​​point cloud to obtain a fitted surface. The controller identifies the uphill starting point based on the fitted surface, outputs the distance from the current electric-assist bicycle to the uphill starting point and the uphill slope angle based on the uphill starting point, and determines the control strategy based on the distance to the uphill starting point, the uphill slope angle, and the current riding condition of the electric-assist bicycle. Within a preset distance before reaching the uphill turning point, the controller controls the electric-assist bicycle to automatically shift gears based on the control strategy.

[0123] The electric-assist bicycle of this application incorporates visual perception into its automatic gear shifting decision-making. It acquires environmental images of the road ahead via a camera, performs depth estimation on these images to obtain road target point cloud data, and combines this with pose data acquired by an inertial sensor. The target point cloud data is then converted to a gravity-aligned coordinate system and fitted with a surface. Based on the fitted surface, the uphill turning points, slope angles, and distances to these turning points are pre-identified and estimated, allowing gear shifting to be completed before the vehicle reaches the uphill turning point. This solution enables gear shifting to be completed in advance under conditions of sufficient speed and stable transmission system load, avoiding the risk of low-speed, high-load shifting shock and chain damage caused by delayed gear shifting. It also eliminates the problem of riders struggling to start after losing speed before an incline, achieving a safe and smooth uphill transition and significantly improving the riding experience.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A shifting method of an electrically assisted bicycle, the electrically assisted bicycle comprising a visual sensor for acquiring real-time environmental images of a riding direction and an inertial sensor for acquiring real-time pose data of the electrically assisted bicycle, characterized in that, The method includes: Based on the environmental image, target point cloud data is obtained; Based on the pose data and the extrinsic parameter calibration relationship between the visual sensor and the inertial sensor, a rotation matrix is ​​constructed from the camera coordinate system to the gravity-aligned coordinate system; based on the rotation matrix, the target point cloud data is transformed into the gravity-aligned coordinate system. Under the gravity-aligned coordinate system, the target point cloud data is fitted with a surface to obtain a fitted surface; Based on the fitted surface, the uphill inflection point of the fitted surface is determined; and based on the uphill inflection point, the slope angle between the uphill surface and the horizontal plane, and the distance between the uphill inflection point and the inflection point of the current electric-assisted bicycle are determined. Based on the slope angle, determine the target gear. Based on the target gear, the distance to the turning point, and the current riding conditions of the electric-assisted bicycle, a control strategy is determined; and within a preset distance before reaching the uphill turning point, the electric-assisted bicycle is automatically shifted according to the control strategy, so that the electric-assisted bicycle switches to the target gear before reaching the uphill turning point.

2. The method according to claim 1, characterized in that, Based on the environmental image, the target point cloud data is obtained as follows: Depth estimation is performed on each pixel in the environmental image to obtain a depth image corresponding to the environmental image; Based on the pixel coordinates of each pixel in the environmental image, the depth value of each pixel in the depth image, and the camera intrinsic parameter matrix, each pixel is transformed from the pixel coordinate system to the camera coordinate system to obtain the original point cloud data. The original point cloud data is used to extract road regions to obtain target point cloud data.

3. The method according to claim 2, characterized in that, The original point cloud data is used to extract road regions to obtain target point cloud data, including: The environmental image is semantically segmented based on a semantic segmentation network to obtain a binary segmentation mask for the road region. The original point cloud data is extracted based on the binary segmentation mask to obtain the target point cloud data; Determine whether the target point cloud data meets the preset accuracy; If not, then a region of interest is selected from the depth image based on spatial geometric prior rules, and the original point cloud data is extracted based on the region of interest to obtain the target point cloud data.

4. The method according to claim 1, characterized in that, The Y-axis of the gravity-aligned coordinate system points in the direction of riding along the horizontal plane, the Z-axis points directly upward along the opposite direction of gravity, and the X-axis points to the right of the direction of riding along the horizontal plane.

5. The method according to claim 4, characterized in that, The target point cloud data is fitted with a surface to obtain the fitted surface, which includes: The target point cloud data is divided into several vertical slices at equal intervals along the Y-axis direction of the gravity-aligned coordinate system. A plane fitting method is used to fit each of the longitudinal slices into a local sub-plane; By splicing the local sub-planes along the riding direction, a fitted surface is obtained.

6. The method according to claim 5, characterized in that, Based on the fitted surface, the uphill inflection point of the fitted surface is determined as follows: Extract the normal vectors of each of the local sub-planes; Based on the normal vectors of each adjacent local sub-plane, the deflection angle of each adjacent local sub-plane is determined; If the deflection angle is greater than a preset angle threshold, the slope inflection point is determined based on the intersection line of the adjacent local sub-planes; and based on the normal vector of the adjacent local sub-planes, it is determined whether the slope inflection point is an uphill inflection point.

7. The method according to claim 5, characterized in that, Determining the slope angle between the uphill surface and the horizontal plane based on the uphill inflection point includes: Based on the normal vector of the local sub-plane after the uphill inflection point, a continuous local sub-plane is determined; The upslope surface is determined based on the continuous local sub-planes; The slope angle between the upslope surface and the horizontal plane is determined based on the average normal vector of the continuous local sub-planes.

8. The method according to claim 1, characterized in that, Based on the fitted surface, the uphill inflection point of the fitted surface is determined as follows: Differentiating the fitted surface yields the slope variation curve along the cycling direction; Based on the slope change curve, the upslope inflection point of the fitted surface is obtained.

9. The method according to claim 1, characterized in that, The cycling conditions include the current gear, current speed, and current cadence.

10. An electric-assisted bicycle, characterized in that, The device includes a vision sensor, an inertial sensor, and a controller, wherein the controller is connected to the vision sensor and the inertial sensor to implement the method of any one of claims 1 to 9.

Images