A lifting or handling vehicle and a method for estimating a maximum speed and acceleration for a vehicle

Abstract

A lifting or handling vehicle (1), in particular a forklift or a telehandler or an articulated loader or an aerial working platform, the vehicle (1) comprising: - at least one attachment or tool (10); - at least one Inertial Measurement Unit, IMU (3) and / or an odometry unit (6) configured to determine speed of the vehicle (1) and / or one or more cameras (5) arranged on board of the vehicle (1) and configured to acquire images and / or videos of areas around the vehicle (1); - one or more control units (2); - a Machine Learning, ML, model (7) embedded in one of the control units (2) and configured to estimate ground conditions in response to signals detected by the IMU (3) and / or the cameras (5) and / or the odometry unit (6); - a behavioral limitation unit (9) embedded in one of the control units (2) and configured to estimate a maximum speed and acceleration in response to the estimated ground conditions.

Description

[0001] The present invention relates to a lifting or handling vehicle and to a method for estimating a maximum speed and acceleration for a vehicle.

[0002] In particular, the vehicle can be a forklift, a telehandler, an articulated loader, an aerial working platform, a cherry picker.

[0003] The invention proposed here may be used in different fields, such as industry, agriculture, construction and maintenance, mining, transports. The proposed vehicle is intended either for lifting people or for handling materials.

[0004] In off-road vehicles such as forklifts, telehandlers, and aerial working platforms, sudden changes in speed and / or acceleration and / or impacts during transport may result in damages on the transported load. These situations are significantly critical in case of fragile or unstable loads. Indeed, when a lifting or handling vehicle travels over an uneven terrain, tilting can occur depending on the displacement of the mass center of gravity of the attachment (i.e., working platform, bucket, forks, etc.).

[0005] In particular, the following situations need to be considered and properly addressed: sudden changes in speed (i.e., acceleration or braking); sudden tool movements (i.e., during rotation, inclination, telescoping, lifting); ground artifacts or defects (i.e., bumpy ground, mud, hardened mud, potholes, sidewalks, etc.)

[0006] These situations can occur either in autonomous driving vehicles or in case of inexperienced operators driving the vehicles.

[0007] In some solutions according to the state-of-the-art vehicles are subject to speed restrictions depending on the ground location.

[0008] Document JP 6690514 B2 discloses an industrial vehicle which is able to suppress a decrease in workability and can impose a vehicle speed limit without requiring additional infrastructure.

[0009] The vehicle (i.e., a forklift) includes a traveling motor, a travel control device controlling the traveling motor, a main controller, an imaging device for imaging a road surface positioned in a travelling direction, and a color discrimination device. The color discrimination device determines a road surface color and outputs a color signal according to the road surface color to the main controller. The main controller sets a speed limit according to the road surface color. The travel control device controls the traveling motor so as to prevent the actual vehicle speed of the forklift from exceeding the speed limit.

[0010] Nevertheless, limiting the performance of the vehicles when loads are not fragile or unstable is not acceptable.

[0011] Document US 2021 / 0122620 A1 discloses an apparatus for controlling a working platform having at least one distance sensor assigned to a wheel of the working platform. A time-of-flight of a measuring beam, which represents a beam emitted obliquely onto a track of the working platform by the distance sensor, is determined. The time-of-flight determined in this manner is compared with a reference time-of-flight in order to determine a change in the inclination angle of the working platform. A control signal for controlling the working platform is output on the basis of the inclination angle change. As a result, a change in the gradient of a track in front of each wheel is individually determined before the particular wheel travels on the track section with the inclination change.

[0012] Document EP 4030393 B1 discloses a method and an apparatus for detecting a bumpy region of a road surface. The method envisages the following steps: acquiring driving state data of a vehicle and an orientation of the vehicle; determining, according to the driving state data, whether the vehicle passes through the bumpy region of the road surface; in a case where the vehicle passes through the bumpy region of the road surface, determining a location of the bumpy region of the road surface according to the driving state data and the orientation of the vehicle.

[0013] According to this solution, the bumpy region of the road surface can be accurately located, thereby effectively reminding other vehicles to take measures in time and reminding road maintenance personnel to carry out road maintenance in time.

[0014] In this context, the object of the present invention is to provide a lifting or handling vehicle and a method for estimating a maximum speed and acceleration for a vehicle, which overcome the problems of the prior art cited above.

[0015] In particular, the object of the present invention is to propose a lifting or handling vehicle which is able to drive taking into account in real-time internal and external factors (i.e., sudden changes in speed and / or tool movements and / or ground artifacts), at the same time avoiding to excessively limit the performance in case of normal loads.

[0016] The stated technical task and specified aims are substantially achieved by a lifting or handling vehicle, in particular a forklift or a telehandler or an articulated loader or an aerial working platform, the vehicle comprising: at least one attachment or tool; at least one Inertial Measurement Unit, IMU and / or an odometry unit configured to determine speed of the vehicle (1) and / or one or more cameras arranged on board of the vehicle and configured to acquire images and / or videos of areas around the vehicle; one or more control units; a Machine Learning, ML, model embedded in one of said control units and configured to estimate ground conditions in response to signals detected by the IMU and / or the cameras and / or the odometry unit; a behavioral limitation unit embedded in one of said control units and configured to estimate a maximum speed and acceleration in response to the estimated ground conditions.

[0017] Further embodiments are defined by the appended dependent claims.

[0018] Further characteristics and advantages of the present invention will more fully emerge from the non-limiting description of a preferred but not exclusive embodiment of a lifting or handling vehicle and a method for estimating a maximum speed and acceleration for a vehicle, as illustrated in the accompanying drawings in which: figure 1 illustrates a lifting or handling vehicle; figure 2 schematically illustrates the block diagram of the lifting or handling vehicle, according to the present invention, communicating with a server; figure 3 schematically illustrates an embodiment of part of the vehicle of figure 2; figure 4 schematically illustrates another embodiment of part of the vehicle of figure 2; figure 5 illustrates the structure of a Deep Neural Network used for the Machine Learning Model of figure 4; figure 6 schematically illustrates an embodiment of part of the vehicle of figure 2, with focus on the behavioral limitation unit; figure 7 illustrates the flow chart of a method for estimating a maximum speed and acceleration for a vehicle, according to the present invention.

[0019] With reference to the drawings, number 1 denotes a lifting or handling vehicle, in particular an off-road vehicle.

[0020] In this context, the lifting or handling vehicle 1 is also shortly referred to as "vehicle".

[0021] For example, the vehicle 1 can be a forklift, a telehandler, an articulated loader, an aerial working platform, a cherry picker.

[0022] The vehicle 1 comprises at least one attachment 10. In this context, an attachment or tool may be of different types, such as a platform, a bucket, a skip, forks or grabs, clamps, jibs, etc.

[0023] The vehicle 1 comprises at least one control unit 2 installed on board of the vehicle 1. In this context, the expression "control unit" is referred to a generic control unit that is configured to direct the operations of the vehicle 1 or to perform partial computations. For example, the control unit 2 may be a microprocessor or an embedded computer.

[0024] The control unit 2 is configured to communicate with a server 20, for example a remotely located server. The communication may be established by any type of connection or network, for example a cloud connection.

[0025] Preferably, the vehicle 1 comprises a plurality of control units 2.

[0026] The control units 2 may be of different type, for example consisting in microprocessors or embedded computers.

[0027] Each control unit 2 may either communicate directly with the server 20 or with one or more of the remaining control units 2.

[0028] Preferably, the vehicle 1 comprises at least one Inertial Measurement Unit 3 which is located on board of the vehicle 1 and is configured to communicate with one or more of the control units 2.

[0029] An Inertial Measurement Unit (shortly referred to with the acronym "IMU") is an electronic device able to measure and report acceleration, orientation, angular rates, and other gravitational forces.

[0030] In this context, the at least one IMU 3 is also referred to as "first IMU".

[0031] In the current invention, the first IMU 3 provides at least six degrees of freedom. In particular, the first IMU 3 comprises three accelerometers and three gyroscopes.

[0032] According to one embodiment of the invention, the vehicle 1 comprises also a second IMU 4 which is installed on board of the attachment 10 of the vehicle 1 and is configured to communicate with one or more of the control units 2.

[0033] For example, the second IMU 4 is located on the forks or on the platform. Preferably, the vehicle 1 comprises one or more cameras 5 configured to communicate with one or more of the control units 2.

[0034] Each camera 5 is arranged on board of the vehicle 1 in a predefined position and with a predefined orientation.

[0035] Each camera 5 is configured to acquire images and / or videos of an area close to the vehicle 1. For example, the cameras 5 may be positioned on the vehicle 1 so as to acquire images and / or videos of any of the following area: a front area, a rear area, a side area of the vehicle 1.

[0036] The cameras 5 may be of different types, for example passive (i.e., monocular, stereo, omnidirectional), or active (lidar, time-of-flight, RGB-depth), or hybrid.

[0037] In some embodiments, the vehicle 1 further comprises an odometry unit 6 configured to determine the speed of the vehicle 1.

[0038] The odometry unit 6 is configured to communicate with one or more of the control units 2.

[0039] As it will be clear later on, the odometry unit 6 is either physical, meaning it comprises sensors 61, or is a software module.

[0040] The vehicle 1 further comprises a Machine Learning model 7 configured at least to estimate ground conditions in response to signals detected by the first IMU 3 and / or odometry unit 6.

[0041] In this context, the Machine Learning model 7 is shortly referred to as "ML model".

[0042] Preferably, the ML model 7 is part of one of the control units 2.

[0043] In one embodiment of the claimed invention the ML model 7 is configured to estimate ground conditions in response only to signals detected by the first IMU 3.

[0044] In another embodiment of the claimed invention, the ML model 7 is configured to estimate ground conditions in response to signals detected by both the first IMU 3 and the odometry unit 6.

[0045] In another embodiment of the claimed invention, the ML model 7 is configured to estimate ground conditions in response to images and / or videos detected by the cameras 5.

[0046] According to an embodiment of the invention, illustrated in figure 3, the odometry unit 6 comprises a plurality of sensors 61 operatively active to measure linear or angular speed of the vehicle 1.

[0047] In particular, the sensors 61 of the odometry unit 6 are configured to detect wheel or motor speed and / or wheel angle.

[0048] For example, the sensors 61 are of the optical or inductive or magnetic type, such as speed or angle encoders.

[0049] Preferably, the ML model 7 is configured to estimate ground conditions in response to signals detected by the first IMU 3 and to signals detected by the sensors 61.

[0050] The signals detected by the first IMU 3 are in particular representative of vibrations and / or oscillations of the vehicle 1, in particular corresponding to displacement speed and depending on the ground conditions.

[0051] Preferably, the signals detected by the first IMU 3 and the signals detected by the sensors 61 are pre-processed by corresponding filtering units 8a, 8b. In particular, the filtering units 8a, 8b are configured to remove noise from the signals coming from the first IMU 3 and from the signals coming from the sensors 61.

[0052] The ML model 7 receives the filtered signals from the filtering units 8a, 8b. In particular, filtered measurements from the sensors 61 and from the first IMU 3 are processed to extract features such as the amplitude of oscillations in pitch (and optionally in roll), the frequency of the oscillations, the energy spectrum at certain frequencies, etc.

[0053] In one embodiment, these extracted features are used by a behavioral limitation unit 9 for producing an estimate of the maximum speed and acceleration that the vehicle 1 can allow.

[0054] As it is will be better explained later, in one embodiment the behavioral limitation unit 9 is also an ML unit.

[0055] All together, the ML model 7 and the behavioral limitation unit 9 are previously trained using supervised or semi-supervised learning.

[0056] For this purpose, the ML model 7 receives recorded measurements from the sensors 61 and from the first IMU 3, as well as labelled time signals. The ML model 7 can be trained to estimate ground conditions using data collected from the first IMU 3 and from the sensors 61 of the odometry unit 6 (when available) while a human operator drives the vehicle 1 at the maximum speed for different types of ground condition.

[0057] Manual labelling of the ground conditions can be performed by the operator after recording the training data. In particular, labelling includes the maximum speed and acceleration that is recommended by the operator for the recorded training data.

[0058] Alternatively, automatic labelling may be performed. Also, synthetic or artificial data can be generated from multiphysical simulation of the vehicle 1 rolling on different types of ground conditions. These synthetic data can be used to train the ML model 7.

[0059] Preferably, training is performed offline and off board of the vehicle 1, for example by a computer or a GPU, or in a server or on the cloud.

[0060] Preferably, during the training phase, different types of ML models are trained in parallel. For example, the following ML models are trained in parallel: Gaussian Process Regression (or other regression algorithms) applied on the filtered signals, and / or Naive Bayes Classifiers applied on extracted characteristics, and / or combination with Decision trees.

[0061] Then, a model selection technique using known criteria is applied to choose the best ML model for a given application, such as industry, agriculture, construction and maintenance, mining, transports.

[0062] For examples, known criteria used for model selection take into account the required computing power and / or common ML performance metrics such as accuracy and sensitivity.

[0063] Another embodiment of the invention is illustrated in figure 4. In this embodiment, the cameras 5 are configured to acquire images and / or videos of areas close to the vehicle 1. The acquired images / videos are used to estimate the ground conditions.

[0064] Preferably, the acquired images are also a basis for estimating the speed of the vehicle 1.

[0065] In particular, in this embodiment the odometry unit 6 is configured to perform visual odometry starting from the images acquired by the cameras 5. In this embodiment, the odometry unit 6 is configured to process images or videos acquired by the cameras 5 to estimate odometry, that means resulting in odometry signals obtained from processing the images and / or videos.

[0066] In particular, the odometry unit 6 is configured to estimate the motion of the one or more cameras 5 in real time using sequential images and / or video acquired by the cameras 5 (i.e., ego-motion).

[0067] In practice, in this embodiment the images from the cameras 5 are used to determine the ground conditions as a first goal and, together with the odometry unit 6 are used to obtain information about where the vehicle 1 is going as a second goal.

[0068] In particular, the odometry unit 6 is a software module which is part of one of the control units 2. For example, the odometry unit 6 is a software module residing in an embedded GPU of one of the control units 2. Preferably, the images and / or videos acquired by the cameras 5 are pre-processed before being fed to the ML model 7.

[0069] Pre-processing consists of normalization, sub-sampling to obtain a specific resolution, while applying a relevant filter to prevent aliasing effects. Additionally, not all of the images and / or videos are used.

[0070] Preferably, for each image a specific region is cropped, and the analysis is restricted to this region of interest.

[0071] In particular, cropping is performed by identifying the region of the image that corresponds to the path that the wheels of the vehicle will travel.

[0072] The odometry signals are also filtered to remove noise.

[0073] In this embodiment, the ML model 7 is configured to estimate ground conditions in response to signals coming from the cameras 5 and optionally from the odometry unit 6, i.e., pre-processed images and / or videos.

[0074] In this embodiment based on the use of the cameras 5, the ML model 7 does not need the presence of the first IMU 3. The odometry unit 6 is sufficient to determine odometry signals for feeding the ML model 7.

[0075] The ML model 7 is previously trained using supervised learning. In particular, images or videos retrieved by the cameras 5 are pre-processed to extract features used to train the ML model 7.

[0076] The same pre-processing is applied for the inputs when performing the estimation in real time and when training the ML model 7.

[0077] The labelling can be performed manually by an operator or automatically. Preferably, a portion of the available dataset (for example, 70%) is used for the training phase, whereas the rest of the available dataset (for example, 30%) is kept for validation. The balance between classes (number of images per class used for training and validation) is verified to avoid bias.

[0078] Preferably, training is performed offline and off board of the vehicle 1, for example by a computer or a GPU, or in a server or on the cloud.

[0079] In this embodiment, the ML model 7 is preferably a Deep Neural Network (DNN).

[0080] The basic structure of the Deep Neural Network used in this embodiment is illustrated in figure 5. The DNN comprises a convolution layer, followed by an activation layer, such as ReLU (Rectified Linear Unit).

[0081] Optionally, the activation layer is followed by a pooling layer, then an upsampling or up-convolution layer, and finally by a softmax layer.

[0082] There can be one or more layers for each of these operations depending on the complexity of the environment in which the vehicle 1 is expected to operate. All these operations applied sequentially allow the DNN to learn spatial information combining characteristics extracted at different scales and levels of abstraction.

[0083] The convolution layer is used to extract patterns, structures, and important characteristics from the images and / or videos. The extracted characteristics then become filters that can be used in parallel to learn how to recognize things (such as a texture, for example) with various levels of abstraction.

[0084] The ReLU layer is used to introduce non-linearities within the DNN, which is essential to model (learn) complex relations between the extracted characteristics.

[0085] The pooling layer reduces the dimensionality of extracted characteristics while preserving the important information. This allows to reduce the amount of required memory and computing power, while aiding the DNN to capture robust characteristics (invariant to small position variations). The upsampling or up-convolution layer is used to restore the spatial resolution of extracted characteristics (the resolution was reduced by pooling operations).

[0086] Finally, the softmax layer is used to transform a score vector into a probabilistic distribution in order to obtain a multiclass classification, where the DNN is able to predict the class of each pixel from a list of possible classes.

[0087] The vehicle 1 also comprises a behavioral limitation unit 9 which is configured to estimate the maximum speed and acceleration in response to estimated ground conditions (which are external conditions). Depending on the specific application, there can be a small or a large number of classes for ground conditions.

[0088] Thus, the behavioral limitation unit 9 may have a different structure.

[0089] In particular, ground conditions include presence of artifacts or defects, such as potholes, sidewalks, bumpy ground, mud, hardened mud, other unstabilized ground, etc.

[0090] In one embodiment, the behavioral limitation unit 9 comprises a look-up table embedded in one of the control units 2.

[0091] The look-up table 9 is used in applications where the number of classes is limited (e.g., a factory).

[0092] In one embodiment, the look-up table 9 is used to determine a maximum speed and acceleration as a function of the ground conditions previously estimated using the images and / or as a function of ground conditions previously estimated using the vibrations / oscillations.

[0093] In the case where the two ground condition estimations are used simultaneously, if these two estimations disagree, the worst-case result is chosen in order to protect the load.

[0094] In addition, as it will be better explained later, the look-up table 9 may also be configured to validate or reduce the maximum speed and acceleration in response to the fragility or instability of the load, which are internal conditions.

[0095] In this case, the look-up table 9 is two-dimensional, so as to further reduce the maximum speed and acceleration for the vehicle 1 in response to fragility / instability of the load.

[0096] As the number of possible ground conditions and possible load fragility levels increase, programming the look-up table 9 can become complex for the operator to identify and to ensure the optimum level of performance. For example, this may occur when the vehicle 1 is used in a farm or an orchard, where the number of classes is larger.

[0097] In this case, the behavioral limitation unit 9 preferably comprises a Neural Network (shortly "NN"), or another ML method, which has been trained to learn the driving behaviour of an expert driver.

[0098] The NN 9 is embedded in one of the control units 2.

[0099] In particular, the NN 9 is trained using supervised learning.

[0100] In order to do this, the expert driver drives the vehicle 1 in the different available ground conditions, during which data are collected on the driver behavior (acceleration, braking, speed, steering), as well as the corresponding ground conditions estimated by the ML model 7 from vibration or images as previously described.

[0101] The NN 9 is preferably trained offline and off board of the vehicle 1, for example by a computer or a GPU, or in a server or on the cloud.

[0102] About the training data, the dataset used contains the ground conditions estimated by both the ML model 7 (either from vibration or images or both, depending on the embodiment), as well as the driver behavior: user controls activation (accelerator & brake pedals), machine speed & steering, and optionally data obtained from the first IMU 3 or second IMU 4.

[0103] During the training process, a portion of the available dataset (for example, 70%) is used for training, while the remaining part of the data is kept for validation. The balance between classes is verified (time duration per class used for training and validation) to avoid bias.

[0104] For example, the behavior of an expert driver is recorded for a full season by collecting signals from the sensors 61 or images from the cameras 5, speeds and accelerations performed by the expert driver, estimations of previously described ML model 7 that give the ground conditions and the estimation of the load fragility / instability.

[0105] These recordings can then be pre-processed to identify segments of the signals where the speed is constant in order to extract the maximum speed.

[0106] The segments where the speed is not constant can be used to extract the maximum acceleration.

[0107] All extracted information can be used to train the NN 9 to produce the same behavior as the expert operator driver.

[0108] In practice, in this case the NN 9 is arranged in cascade to the ML model 7 so as to determine the maximum speed and acceleration.

[0109] Optionally, fragility and / or instability of the load is also considered to further reduce the estimated maximum speed and acceleration of the vehicle 1.

[0110] In this context, the characteristics of the load carried by the vehicle 1, such as the degree of fragility and / or instability of the load, is indicated as "load figure" and is identified here as LF.

[0111] In particular, the load figure LF can be obtained from a text label or from a color code applied to the load. Alternatively, the load figure LF may be determined by identifying the content of a pallet.

[0112] Alternatively, the load figure LF can be inserted by an operator through a remote or local operational interface (e.g., console or smart device, a cabin button or selector).

[0113] The load figures LF are available to one or more of the control units 2. Optionally, the load figures LF are memorized in the server 20 or in a fleet management system in communication with one of the control units 2.

[0114] In this embodiment, the behavioral limitation unit 9 is configured to validate or reduce the estimated maximum speed and acceleration of the vehicle 1 in response to the load figure LF received by one of the control units 2.

[0115] In particular, validating or reducing the estimated maximum speed and acceleration may be provided as an option for the behavioral limitation unit 9, which may be used or not.

[0116] In particular, the behavioral limitation unit 9 is configured to receive an input command from an operator, in response to which the behavioral limitation unit 9 is configured to activate or deactivate the validating / reducing option.

[0117] For example, the input command may be inserted by an operator through a remote or local operational interface (e.g., console or smart device). The further limitation caused by the load figure LF can be discrete or continuous.

[0118] According to one aspect, the load figure LF is also used to limit any the following actions or operations of the vehicle 1 or of the attachment 10: vehicle braking, vehicle steering, vehicle tower rotation speed and acceleration, tool lifting speed and acceleration, tool telescoping speed and acceleration, tool rotation speed and acceleration, tool inclination speed and acceleration, accessory activation (e.g., clamps closing force, tool rotation speed and acceleration).

[0119] Finally, other restrictions may be applied to the vehicle 1 or to the attachment 10 based on country regulations, safety regulation and performance.

[0120] As already anticipated, where the behavioral limitation unit 9 is in the form of a look-up table, the latter may have two dimensions in case of a fragile or instable load, so as to validate or reduce the maximum speed and acceleration for the vehicle 1.

[0121] According to one embodiment, the behavioral limitation unit 9 is configured to apply an interpolation function or matrix in response to the load figure LF, so as to impose one or more restrictions to the vehicle 1 or to the attachment 10.

[0122] In practice, the load fragility or instability is optionally used to further restrict the initial behavioral limitations estimated on the basis of the estimated ground conditions.

[0123] Some examples are provided below to clarify the invention.

[0124] In a first example, there are four types of ground conditions: 1) flat; 2) stabilised; 3) stabilised but damaged (bumps and holes present or expected); 4) dirt (unstabilised / changing).

[0125] There is no further restriction due to fragility or instability of the load. Thus, the maximum speed and acceleration of the vehicle 1 need to be determined for each type of ground conditions.

[0126] In a second example, in addition to the four types of ground conditions there are also three levels (or degrees) of fragility for the load, which are: 1) no load 2) normal load 3) fragile load

[0127] In this second example, the situation is mapped by a table or matrix of 3 x 4 (= 12) possibilities and the operator needs to define the maximum speed and acceleration for each of cell of the table.

[0128] In a third example, in addition to the four types of ground conditions there is imposed a limitation of the speed for each ground condition depending on the weight of the load.

[0129] In this third example, for each type of ground condition there is a curve of speed vs. weight, for a total of four curves. These curves could be further offsetted by a "load fragility coefficient".

[0130] In a fourth example, in addition to the four types of ground conditions there are imposed limitations as follows: continuous ground damage condition: measured as a level of pitch or yaw oscillation or oscillation energy of the machine; limitation of the speed for each ground condition depending on the weight of the load.

[0131] In this fourth example, there is a 3D graph speed vs. weight vs. ground condition. The graph could be further offsetted by a "load fragility coefficient".

[0132] The vehicle 1 can be optionally equipped with special mechanisms that allow it to face rough ground conditions, for example a mechanism allowing to select between 2WD (2 wheels drive) and 4WD (4 wheels drive) or the activation of a mast or boom suspension.

[0133] The activation of this mechanism (in the example, the selection between 2WD and 4WD) is recorded as part of the operator behavior when recording the dataset for behavioral training.

[0134] In this case, the NN 19 outputs the activation / deactivation of the mechanisms in addition to the behavioral restrictions.

[0135] The vehicle 1 may be optionally equipped with means of localisation (such as GPS, LiDAR, etc.).

[0136] A cartography containing information on the ground conditions may be stored on the server 20 or in a memory on board of the vehicle 1.

[0137] The cartography augmented with the ground conditions and the fragility / instability data of the load is taken into consideration for computing the fastest route between the current vehicle position and a target position when a transport mission is requested.

[0138] Additionally, the vehicle 1 (or the remote server 20) is equipped with an algorithm or alert mechanism that allows to warn an operator or a site manager of important degradations in ground conditions, for example by comparing the change of ground conditions with respect to a reference ground condition. The reference ground condition can be reset by an operator or site manager, for example when the infrastructure or road maintenance is performed.

[0139] Two further embodiments (not shown in the accompanying drawings) are also possible, which have been devised to address a problem arising in the mining technical field.

[0140] In the mining field, among other tasks, vehicles like the one of the invention mount clamps as the attachment 10, which are used to drive tubes into the ground. There is a number of different tubes with different structural resistance.

[0141] Therefore, on the one side, if the grip strength with which the clamps hold the tube is excessive for that kind of tube, the tube might break. On the other side, if the grip strength is too low, the clamps may lose the grab and the tube would fall.

[0142] According to a first embodiment, one of the control units 2 is configured to limit the grip strength of the clamps according to maximum strength values selected or set up by the operator.

[0143] Preferably, the control unit 2 is connected to or provided with a user interface configured to enable the operator to choose the desired maximum strength values from a pre-recorded set. To different types of tubes correspond different maximum strength values.

[0144] In this way, when the operator operates the clamps, the actuator which open and close them cannot apply a grip strength which is above the maximum value chosen, thereby preventing the above-mentioned drawbacks of the prior art.

[0145] According to the second embodiment, the vehicle 1 is provided with at least one recognition device for detecting the kind of tube to be grasped and connected with one or more of the control units 2. For example, the recognition device can be a camera 5.

[0146] In this case, the control unit 2 is provided with an ML model configured to assess which type is the tube to be grasp and configured to automatically control the strength applied by the clamps to grasp the tube according to the assessed type.

[0147] Moreover, in both embodiments, in order to enable the control unit 2 to realize a feedback control, the clamps can be provided with at least a sensor able to measure the grip strength of the clamps and configured to communicate with one or more of the control units 2. The sensor can be a strain gauge or a pressure sensor for measuring a pressure into a hydraulic actuator of the clamps or another suitable sensor.

[0148] A computer-implemented method for estimating a maximum speed and acceleration for the vehicle is disclosed herewith.

[0149] The method, indicated with number 100, comprises at least the following steps: detecting signals by means of the first IMU 3 and / or odometry unit 6 and / or cameras 5 (this step is indicated with 101); estimating ground conditions in response to the detected signals (this step is indicated with 102); estimating a maximum speed and acceleration for the vehicle 1 in response to the estimated ground conditions (this step is indicated with 103).

[0150] According to one embodiment, the step of detecting 101 is carried out at least by the first IMU 3.

[0151] Preferably, the step of detecting 101 is also carried out by the sensors 61 of the odometry unit 6, which are operatively active to measure linear or angular speed of the vehicle 1.

[0152] Optionally, also signals detected by the second IMU 4 applied to the attachment 10 may be used in the step1 01.

[0153] The step of estimating the ground conditions 102 is carried out by the ML model 7, which has been previously trained as explained in the description above.

[0154] In particular, as already explained above, different types of ML models are trained in parallel, the best ML model for the specific application is selected and used for step 102.

[0155] According to another embodiment, the step of detecting 101 is carried out using the cameras 5 and optionally the odometry unit 6 performing visual odometry on the basis of images and / or videos acquired by the cameras 5 or using traditional odometry sensors.

[0156] The step of estimating the ground conditions 102 is carried out by the Deep Neural Network 7, which has been previously trained as described above.

[0157] The maximum speed and acceleration are then estimated on the basis of the estimated ground conditions (step 103) by the behavioral limitation unit 9, either in the form of a look-up table or an ML model.

[0158] Optionally, the method 100 comprises also a step of validating or reducing the estimated maximum speed and acceleration in response to the load figure LF. This step is indicated with number 104.

[0159] The characteristics of the lifting or handling vehicle and of the method for estimating a maximum speed and acceleration for a vehicle, according to the present invention, are clear, as are the advantages.

[0160] In particular, thanks to the ML model that is able to estimate ground conditions in response to different signals (i.e., detected by IMU, odometry sensors or cameras) it is possible to estimate in real time a maximum speed and acceleration for the vehicle using either a look-up table or machine learning algorithms such as a Neural Network.

[0161] In addition, the fragility or instability of the load is optionally considered for providing a further restriction to the maximum speed and acceleration, as well as to other actions or operations.

Claims

1. A lifting or handling vehicle (1), in particular a forklift or a telehandler or an articulated loader or an aerial working platform, said vehicle (1) comprising: - at least one attachment or tool (10); - at least one Inertial Measurement Unit, IMU (3) and / or an odometry unit (6) configured to determine speed of the vehicle (1) and / or one or more cameras (5) arranged on board of the vehicle (1) and configured to acquire images and / or videos of areas around the vehicle (1); - one or more control units (2); - a Machine Learning, ML, model (7) embedded in one of said control units (2) and configured to estimate ground conditions in response to signals detected by the IMU (3) and / or the cameras (5) and / or the odometry unit (6); - a behavioral limitation unit (9) embedded in one of said control units (2) and configured to estimate a maximum speed and acceleration in response to the estimated ground conditions.

2. The vehicle (1) according to claim 1, wherein said ML model (7) is configured to estimate ground conditions only in response to signals detected by the at least one IMU (3).

3. The vehicle (1) according to claim 1, wherein said ML model (7) is configured to estimate ground conditions in response to signals detected by both the at least one IMU (3) and the odometry unit (6).

4. The vehicle (1) according to claim 3, wherein the odometry unit (6) comprises a plurality of sensors (61) configured to measure linear or angular speed of the vehicle (1), said ML model (7) being configured to estimate ground conditions also in response to the linear or angular speed measured by the sensors (61).

5. The vehicle (1) according to claim 4, further comprising filtering units (8a, 8b) configured to filter noise respectively from signals detected by the at least one IMU (3) and from signals detected by the plurality of sensors (61), the ML model (7) receiving filtered signals from said filtering units (8a, 8b).

6. The vehicle (1) according to any one of the claims 2 to 5, wherein the ML model (7) is trained using supervised learning.

7. The vehicle (1) according to any one of claims 2 to 6, wherein the ML model (7) is chosen among one of the followings: Gaussian Process Regression, Decision trees, Naïve Bayes Classifiers.

8. The vehicle (1) according to claim 1, wherein said odometry unit (6) is embedded in one of said control units (2) and is configured to process images and / or videos detected by the one or more cameras (5) to determine odometry signals.

9. The vehicle (1) according to claim 8, wherein images and / or videos acquired by the one or more cameras (5) are pre-processed and cropped.

10. The vehicle (1) according to claim 8 or 9, wherein the ML model (7) is trained using supervised or semi-supervised learning.

11. The vehicle (1) according to any one of claims 8 or 10, wherein the ML model (7) is a Deep Neural Network, DNN.

12. The vehicle (1) according to claim 11, wherein the DNN (7) comprises at least the following layers: a convolution layer, followed by an activation layer, followed by a pooling layer, followed by an upsampling or up-convolution layer, followed by a softmax layer.

13. The vehicle (1) according to any one of the preceding claims, wherein the behavioral limitation unit (9) comprises a look-up table.

14. The vehicle (1) according to any one of the claims 1 to 12, wherein the behavioral limitation unit (9) comprises a Neural Network, NN (9) previously trained to learn the driving behavior of a driver, said NN (9) being arranged in cascade to the ML model (7) so as to determine the maximum speed and acceleration.

15. The vehicle (1) according to claim 14, wherein the NN (9) is trained using supervised or semi-supervised learning.

16. The vehicle (1) according to any one of the preceding claims, wherein the behavioral limitation unit (9) is also configured to validate or reduce the estimated maximum speed and acceleration of the vehicle (1) in response to a load figure (LF) which is representative of a degree of fragility or instability of a load carried by the vehicle (1).

17. The vehicle (1) according to claim 16, wherein the load figure (LF) is also used to limit other actions or operations of the vehicle (1) or of the attachment (10).

18. The vehicle (1) according to claim 16 or 17, wherein the behavioral limitation unit (9) is configured to apply an interpolation function or matrix in response to the load figure (LF).

19. A computer-implemented method (100) for estimating a maximum speed and acceleration for the vehicle (1) according to any one of the preceding claims, said method (100) comprising the following steps: - detecting signals (101) by means of the at least one IMU (3) and / or odometry unit (6) and / or cameras (5); - estimating ground conditions in response to said detected signals (102); - estimating a maximum speed and acceleration for the vehicle (1) in response to the estimated ground conditions (103).

20. The method (100) according to claim 19, further comprising a step of validating or reducing the estimated maximum speed and acceleration of the vehicle (1) in response to a load figure (LF) which is representative of a degree of fragility or instability of a load carried by the vehicle (1).

21. A computer program product having instructions which, when executed by a computer device or system, cause the computing device or system to perform the method according to claim 19 or 20.

Images