Chaining of tyres by robotic means using three-dimensional viewing

Abstract

The invention relates to a process for automatically chaining tyres from a group of tyres which are in an unknown arrangement and for which a predetermined target placement has to be achieved, the chaining process being implemented by at least one processor comprising an image processing module which uses a positioning algorithm to choose the optimal placement of a target tyre to be chained. The invention also relates to a system for automatically chaining tyres, comprising at least one gripping device, a detection system for collecting information about the physical environment around each gripping device, and at least one communication server for executing programmed instructions stored in a memory of one or more processors of the chaining system which uses a positioning algorithm to implement the disclosed chaining process.

Description

[0001] Description

[0002] CHAINING OF TIRES BY ROBOTIC MEANS EMPLOYING THREE-DIMENSIONAL VISION

[0003] Technical Field

[0004] The invention relates to a system that performs a method of chaining tires from a group of tires in an unknown arrangement. More particularly, the invention relates to an automatic system capable of chaining tires in any target location, regardless of the parameters of the target location and the varying parameters of different tires being arranged.

[0005] Context

[0006] In the field of tire chaining, there are tire arrangements that facilitate their handling and ensure their optimal storage in the available storage space. Referring to Figure 1, an embodiment of tire storage is shown in which several layers of tires 10 partially overlap each other. In this type of tire storage (known in the art by the designation "rick-rack"), the tires are stacked (or "chained") in a container 12, the overlapping direction being reversed from one layer to the next according to a chaining process. In this configuration, the space between side portions 12a of the container 12 is used in an optimal manner.The container 12 may be selected from among the containers known for carrying out the transport of tires, including, without limitation, pallets, truck beds, chained trucks, box trucks and their equivalents. The structure of this stacking pattern is described in detail in patent DE2426471A1.

[0007] Other types of tire storage are also known for transporting such tires in containers. In one embodiment of tire storage called "roll storage," the tires are stored side by side on their treads along a common horizontal axis. In another embodiment of tire storage called "stack storage," the tires are stacked side by side on their sidewalls along a common vertical axis. Automated solutions exist for chaining tires into containers depending on the type of storage chosen. These solutions incorporate vision-based robot control for tire gripping.Examples are provided by US8,244,400 (which discloses an automated tire chaining device on a support which comprises a handling device with one or more gripping tools coupled in order to receive and deposit the tires), US8,538,579 (which discloses a depalletizing system for implementing a method of depalletizing tires deposited on a support, the system being guided by a robot with a gripping tool), and US9,440,349 (which discloses an automatic tire loader / unloader for stacking / unstacking them in a trailer, comprising an industrial robot capable of selective articulated movement).

[0008] Chaining technologies often use a combination of laser scanning of the surface assumed to contain the objects to be picked up and knowledge of the desired object (CAD). The system seeks to superimpose the elements measured in real space with the known CAD elements to precisely find the object and its spatial configuration and then be able to grasp it in a way consistent with the gripper design. Thus, the majority of methods widespread in industry work by trying to control the environment. This can be done from a hardware point of view by requiring very specific installations for the task, or by learning references in the fixed working environment, or even by trying to register a CAD model in a point cloud type scene to detect an object.

[0009] A method requiring a specific hardware installation, however sophisticated, will no longer work in the event of significant variations in the installation. A model recalibration requires that all objects in the container be identical (within a scale factor) and largely visible to have a suitable matching. For example, patent US8,538,579 proposes using CAD data of tires to carry out the work of "storage densification". This requires either a completely homogeneous pallet of identical tires whose dimensions are entered once and then the system processes them automatically, or a case-by-case reading of the tire reference, the call for its dimensions in a CAD database, the calculation of the optimal storage position and then its manipulation. In addition, there are also storages specific to tires outside of product recognition.For example, patent EP2062824B1 presents unit boxes that facilitate the transport of cylindrical items such as tires. In particular, a folded box is disclosed that allows the tires to be handled and stored stably. As a result, storing the boxes themselves introduces a significant loss of volume and is expensive.

[0010] The filling and emptying of containers known to carry out the transport of tires (and particularly, trucks) are tasks that are by definition random: the order of the chained tires is not known in advance, nor their size; accessibility is reduced, both for gripping; and the tires can only be seen partially. Controlling the environment is therefore not a viable situation.

[0011] Thus, the disclosed invention makes it possible to establish a link between chaining processes and the management of the chaining of tires arranged in a group of arranged tires without prior knowledge of their precise configuration.

[0012] Summary of the invention

[0013] The invention relates to a method for automatically chaining tires from a group of tires in an unknown arrangement and intended for chaining in a predetermined target location, the chaining method implemented by at least one processor comprising an image processing module which employs a positioning algorithm to choose the optimal location of a target tire to be chained, characterized in that the chaining method comprises the following steps: a step of providing a chaining system of which the processor is a part, wherein the chaining system comprises at least one gripping device which performs the gripping of a target tire of the group of tires; a step of estimating the dimensions of a target location of which the tires of the group of tires are intended for chaining, this step comprising a step of scanning the target location to determine its parameters;a step of creating the origin of a location-device reference mark during which the chaining system creates a reference mark to know the positioning coordinates of a first tire in the target location; a step of estimating the dimensions of a target tire to be chained in the target location, during which the device identifies the first tire to be taken from the group of tires; a positioning step for each tire to be chained, this step comprising the following steps: a step of carrying out a tire positioning process which is carried out for each tire to be chained; a step of approaching the device towards the identified target tire; and a step of taking the identified target tire; a step of scanning the tire after its positioning carried out during the positioning step;and a final tire position updating step, during which the actual position of the last positioned tire, estimated during the positioned tire scanning step, includes the 2D center of the positioned tire and its orientation, thereby updating the positioning algorithm the actual position of the last positioned tire, such that the update makes it possible to check that the estimated actual position is consistent with the physical limits of the target location; and such that the tire chaining is performed when the gripping device releases the tire.;

[0014] In some embodiments of the chaining method of the invention, during the step of performing a positioning process for each tire to be chained: the group of tires is modeled in a two-dimensional (2D) space as a rectangle, wherein the width and height of the rectangle are equal to the width and height of the interior of the target location;and each tire of the tire group is modeled using a rectangle whose width and height are equal to the width and diameter of the tire represented. In some embodiments of the chaining method of the invention, the step of estimating the dimensions of a target tire to be chained comprises a few-shot learning process comprising the following steps: a step of acquiring data corresponding to the arranged tires, during which a detection system of the gripping device captures an initial image of the randomly arranged tires; a step of feeding an extraction neural network and an attention neural network, during which the two neural networks are trained by taking a plurality of sample images obtained during the data acquisition step as training data and a plurality of image object classifications as data labels;a step of performing a three-dimensional (3D) reconstruction process fully performed on the basis of the data of the extraction neural network and the attention neural network, during which coordinates corresponding to the location of an identified target tire and its orientation are reconstructed from these data, so that this reconstruction serves to provide the geometric information necessary to generate an ideal gripping point of the identified target tire; a step of approaching the gripping device towards the identified target tire, during which the attention neural network sends to the gripping device the coordinates corresponding to the location of the identified target tire and its orientation; and a step of outputting the identified target tire from the tire arrangement to put it in the target location.;

[0015] In some embodiments of the chaining method of the invention, the step of feeding the extraction neural network and the attention neural network comprises the following steps: a step of training the extraction neural network to segment a scene seen by the detection system of the management system; and a step of constructing the attention mechanism, during which the attention neural network extracts features differentiated among different categories in a target tire detection model, so that the model is guided to locate key areas in a segmented image; wherein the step of training the extraction neural network comprises a step of segmenting data based on a plurality of cycles of a repetitive movement of the gripping device during one or more chaining cycles.

[0016] In some embodiments of the chaining method of the invention, during the step of performing the 3D reconstruction process, the orientation, dimensions and location of the identified target tire are reconstructed from the data of the extraction neural network and the attention neural network. In some embodiments of the chaining method of the invention: the step of acquiring the data of the few-shot learning process comprises a step of constructing a point cloud from the RGB-D images; and the step of approaching the gripping device comprises a step of gripping the identified target tire at the ideal gripping point calculated during the step of performing the 3D reconstruction process.

[0017] In certain embodiments of the chaining method of the invention, the tire positioning process comprises the following steps: a step of positioning the first tire which can be taken from the group of tires in which this first tire is positioned flat at a place corresponding to the choice of the location-device reference mark created during the step of creating the origin of the reference mark; and a step of arranging the tires starting from the second tire, during which the positioning algorithm uses a notion of row which is a sequence of tires positioned in the same direction, and in which each type of row is associated with a set of orientations to be tested for the tire to be positioned.

[0018] In certain embodiments of the chaining method of the invention, during the step of arranging the tires, the positioning algorithm applies the following steps to select the candidate positions of the tire: a step of first calculating all the positions which put the tire in contact with the limits of the target location; a step of then calculating all the positions which put the tire in contact with the other tires already chained; a step of preserving, among all the positions calculated during the previous step, the positions allowing the tire to have at least two points of contact with the limits of the group of tires or with the other tires; and a last step of preserving the positions allowing the current row to be completed which represents the only position allowing the current row to be completed.In some embodiments of the chaining method of the invention, the step of arranging the tires further comprises: a step of calculating a stability and chaining criterion making it possible to know whether the position of the tire is stable on each position retained at the end of selection of the candidate positions of the tire; and a conversion step during which the 2D center of the tire resulting from the positioning algorithm is converted to the 3D center of the tire in the target location; so that the positioning algorithm indicates the center of the tire in the rear plane of the group of tires as well as its orientation to position the tire in the target location so as to obtain the 3D center of the tire and to couple it with the orientation.

[0019] In certain embodiments of the chaining method of the invention, during the step of calculating a stability and chaining criterion: the stability of the position of the new tire depends on the vertical starting from its center of gravity and the support zone of the new tire; and the chaining criterion is based on a horizontal signed distance between the center of the last tire installed and the end of the new tire.

[0020] In some embodiments of the chaining method of the invention, during the tire scanning step performed during the positioning step: the chaining system scans an area in which the last tire was positioned; and an attention mechanism is then used to detect the last positioned tire.

[0021] The invention also relates to a system for automatically chaining tires from a group of tires in an unknown arrangement and for which a predetermined target location is to be achieved, characterized in that the chaining system comprises: at least one gripping device comprising a robot having a gripping device supported by a pivoting elongated arm and extending from the elongated arm to a free end where a gripper is arranged along a common longitudinal axis; a detection system for collecting information on the physical environment around each gripping device, the detection system comprising one or more sensors configured to perform two-dimensional (2D) and / or three-dimensional (3D) image detection;and a communication network that manages incoming data to the chaining system from at least one gripping device, the communication network comprising at least one communication server for executing programmed instructions stored in a memory of one or more processors of the chaining system that employs a positioning algorithm to implement the disclosed chaining method; such that each cleaning device is configured to one or more parameters of at least one identified tire calculated by an image processing module incorporated in the memory of the processor to set the cleaning device in motion so that the gripping device can perform the gripping of a target tire during the chaining process performed by the chaining system.;

[0022] In some embodiments of the chaining system of the invention, the detection system comprises at least one RGB-D type camera attached to at least one of the gripping device, the elongated arm and the gripper of the cleaning device. In some embodiments of the chaining method of the invention, the predetermined target location comprises at least one location selected from one or more containers, one or more trucks, one or more trailers, one or more warehouses, one or more distribution centers, one or more conveyors, one or more pallets and one or more storage means.

[0023] Other aspects of the invention will become apparent from the following detailed description.

[0024] Brief description of the drawings

[0025] The nature and various advantages of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals designate like parts throughout, and in which:

[0026] [Fig 1] Figure 1 shows a perspective view of one embodiment of tire storage.

[0027] [Fig 2] [Fig 3] Figures 2 and 3 represent components of a known tire in a meridian plane.

[0028] [Fig 4] Figure 4 shows a partial perspective view of one embodiment of a gripping device employed in a chaining system of the invention.

[0029] [Fig 5] Figure 5 shows an example of a group of arranged tires being processed by a gripping device of the type shown in Figure 4.

[0030] [Fig 6] Figure 6 represents a flow diagram of an embodiment of a chaining method of the invention carried out by the chaining system.

[0031] [Fig 7] Figure 7 shows an example of a trailer being used at a target location for which tire chaining is performed during the chaining method of the invention. [Fig 8] Figure 8 shows an example of a first pickable tire being identified for positioning in a target location during the chaining method of the invention.

[0032] [Fig 9] Figure 9 represents a tire positioning process carried out during a positioning step of the chaining method of the invention.

[0033] [Fig 10] Figure 10 shows a schematic view of the types of tire positioning directions achieved during the positioning process.

[0034] [Fig H] Figure 11 shows a schematic view of an orientation assembly to be tested for the tire to be positioned during a tire stowage stage of the positioning process.

[0035] [Fig 12] Figure 12 shows an example of how to arrange tires in a target location.

[0036] [Fig 13] [Fig 14] Figures 13 and 14 represent, respectively, a stable position and an unstable position representing the stability of the position of a positioned tire during the positioning process.

[0037] [Fig 15] [Fig 16] Figures 15 and 16 represent an illustration of a chaining criterion used during the positioning process.

[0038] [Fig 17] Figure 17 shows a schematic view of a conversion step performed during the positioning process.

[0039] [Fig 18] [Fig 19] Figures 18 and 19 represent schematic views, respectively, of an actual position of the last positioned tire and the update of this actual position during the positioning process.

[0040] Detailed description

[0041] When considering the type of tire storage that best utilizes the available storage space, the geometry of the tires being transported must be considered. Figures 2 and 3 show schematics of a tire P that typically includes two circumferential beads intended to allow the tire to be attached to a rim. Each bead includes an annular reinforcing bead. The construction of a tire is typically described by a representation of its constituents in a meridian plane, i.e., a plane containing the tire's axis of rotation. The radial, axial, and circumferential directions respectively denote the directions perpendicular to the tire's axis of rotation, parallel to the tire's axis of rotation, and perpendicular to any meridian plane.The expressions "radially", "axially" and "circumferentially" mean respectively "in a radial direction", "in the axial direction" and "in a circumferential direction" of the tire. The expressions "radially inward" and "radially outward respectively" mean "closer, respectively further, from the axis of rotation of the tire, in a radial direction. Referring to Figure 2, the tire P comprises an inner boundary Fi and an outer boundary FE which together define the boundaries of a sidewall F of the tire P. The inner boundary Fi separates the sidewall F of the tire and a rim (not shown) to which the tire is intended for mounting. The tire P also comprises a rim radius Rj defined as the distance between a center point C of the tire and the inner boundary Fi which separates the rim and the sidewall F of the tire.The tire P also includes an internal sidewall diameter defined as twice the rim radius Rj. The tire P also includes a tire radius Rp defined as the distance between the center point C and an external boundary FE of the sidewall F which represents the tire's rolling surface. The tire P also includes a tire diameter defined as twice the tire radius Rp.

[0042] Referring to Figure 3, the inflated and unloaded tire P includes several parameters of its geometry, including a nominal section width Lp and a height Hp (the height Hp often being expressed as a percentage of the width Lp). The tire P also includes a measurement Dj which represents the diameter of a rim to which the tire is intended for mounting (this measurement being substantially equal to the internal sidewall diameter Fi). It is understood that each of these parameters can be expressed in equivalent known length measurements (for example, in millimeters (mm) or in inches (in)).Referring now to Figures 4-5, in which like numerals identify like elements, Figures 4 and 5 show one embodiment of a gripping device (or "device") 100 that performs the gripping of a target tire of an unknown arrangement of tires and for which a target location is to be achieved during a chaining cycle of the invention. The device 100 is part of a tire chaining system (or "chaining system" or "system") of the invention.

[0043] It is understood that the term "chaining" includes the functions of storing and retrieving arranged (or "chained") tires as well as the target arrangement (including loading and unloading) of the tires. It is understood that the term "target tire" (in the singular or plural) is used herein to refer to a tire that is present in the physical environment of the device 100 and that is identified for positioning during a chaining cycle. It is understood that the term "target location" (in the singular or plural) includes a dedicated space where the tires will be arranged (e.g., a trailer, a belt, a conveyor, a crate, a rack, etc.). The term "target arrangement" (in the singular or plural) includes a desired arrangement for the tires arranged in a target location (e.g., in a "rick-rack" or "chained" manner), "roll-up storage," or "stack storage").

[0044] The system of the invention performs a motion chaining method of the device 100 (or "chaining method") that incorporates a combination of vision techniques to correctly and quickly reconstruct the observed scene from scattered three-dimensional (or "3D") point clouds, derived from a fragmented view of the target tires. The device 100 is usable in spaces where tires are arranged in an unknown manner and in spaces where their target arrangement must be achieved. For example, the device 100 can be used with respect to a crate 200 having tires P200 arranged therein. The device 100 can take the tires P200 arranged in the crate 200 to chain them in one or more target locations (for example, in a trailer 300 as shown in Figure 7). It is understood that any suitable type of container could be used instead of the crate 200.

[0045] The device 100 therefore performs a target arrangement of tires in a predetermined target location. It is understood that the device 100 can operate in several physical environments without knowledge of their parameters in advance (for example, an initial or targeted arrangement of the tires in a truck, in a warehouse, in a distribution center, on a conveyor, on a pallet or relative to other known storage and / or transport means).

[0046] Referring further to Figures 4 and 5, in one embodiment of the device 100, the gripping device comprises a robot of the type disclosed by the Applicant's publication WO2022 / 135968. The robot has a gripping peripheral 104 supported by a pivotable elongated arm 106. The gripping peripheral 104 extends from the elongated arm 106 to a free end 104a where a gripper 108 is disposed along a common longitudinal axis. The robot is set in motion so that the gripper 108 can effect gripping of a target tire by the device 100 during a chaining process performed by the system of the invention (as described below).

[0047] It is understood that the precise configuration of the device 100 shown in Figures 4 and 5 is given by way of example. For example, the device 100 could be provided with several types of grippers depending on the characteristics of a group of tires intended to be chained. In another example, the device 100 may comprise a fixed robot installed at a chaining installation, fixed, for example, to a support relative to which the robot extends. In this case, it is understood that the robot can be fixed to a ceiling, a wall, a floor or any support which allows the chaining method of the invention to be carried out.

[0048] It is understood that the device 100 may comprise at least one roaming robot. By “roaming”, it is understood that the gripping device may be set in motion either by integrated movement means (for example, one or more integrated motors) or by non-integrated movement means (for example, one or more mobile means including autonomous mobile means). It is understood that the gripping device may be a conventional industrial robot or a collaborative robot or even a delta or cable robot.

[0049] The device 100 also includes a sensing system (not shown) for collecting information about the physical environment around the device. The sensing system includes one or more sensors (including one or more cameras) configured to perform two-dimensional (2D) and / or three-dimensional (3D) image sensing, 3D depth sensing, and / or other types of sensing of the physical environment around the gripping device (it is understood that the terms "sensor" and "camera" are used interchangeably). In embodiments of the device 100 of the type shown in Figures 4 and 5, the one or more sensors of the sensing system are attached to at least one of the gripping device 104, the elongated arm 106, and the gripper 108 of the robot.In embodiments of the system of the invention, these sensors could be part of an overall detection system that employs these sensors together with one or more sensors positioned in the physical environment in which the device 100 operates (e.g., one or more cameras 400) (see Figure 5).

[0050] In one embodiment of the device 100, the sensing system includes at least one camera that provides 3D images represented as a set of 3D points with X, Y, Z coordinates, and red, green, blue color values ​​(the "RGB" or "RGB-D" format) (referred to as an "RGB-D camera"). In this embodiment, an RGB-D camera is attached to at least one of the gripping device 104, the elongated arm 106, and the gripper 108 of the device 100. Two or more RGB-D cameras may be oriented to provide a predetermined overlap between the cameras' fields of view. As used herein, the term "camera" includes one or more cameras.

[0051] RGB-D cameras generally provide depth information using depth maps, being images where each pixel contains the distance between the camera and the corresponding point in space. Compared to traditional measurement methods such as manual measurement and other measurements based on electronic devices, 3D point cloud data from RGB-D type cameras has a much higher measurement rate. By using a sparser structure, a point cloud can be constructed from the RGB-D images by calculating the real world (e.g., X, Y, Z coordinates) with the intrinsic data of a scanning camera. Thus, information about the physical environment around the device 100 is obtained from the 3D point cloud data obtained from sensing technologies that are capable of capturing the 3D surface geometries of the target tires accurately and efficiently.The term "point cloud" (singular or plural) is used here to refer to a collection or collections of data points in space. A camera or cameras (or equivalent devices) collect three-dimensional (3D) data and detect the surfaces of objects (e.g., arranged tires) using a series of coordinates. Storing the information as a collection of spatial coordinates can save space because many objects do not fill a large portion of the environment. Even if the information is not visual, interpreting the data as a point cloud helps understand the relationship between multiple variables through classification and segmentation.

[0052] It is understood that one or more cameras may include one or more programming modes, including by learning, to feed, modify, and train at least one neural network. Although the embodiments are described herein with respect to the use of one or more neural networks (e.g., convolutional neural networks or CNNs) as a machine learning model, other types of machine learning models may be used.These include, but are not limited to, models using linear regression, logistic regression, decision trees, support vector machines, naive Bayes, nearest neighbor (knn), K stands for clustering, random forest, dimensionality reduction algorithms, gradient-based algorithms, neural networks (e.g., autoencoders, CNNs, RNNs, perceptrons, logarithmic short-term memory (LSTM), Hopfield, Boltzmann, deep belief, deconvolution, generative adversarial (GAN), etc.) and their complements and equivalents. The one or more CNNs may be trained with ground truth data that is generated using sensor data representative of the motion of the device 100, including the positioning of the gripper 108.

[0053] The detection system of the device 100 detects the presence of a group of tires in the field of view of the detection system (e.g., the field of view of a camera of the device 100), which triggers it to capture the image of a target tire (see, for example, the target tire P200* shown in Figure 5). In all embodiments of the device 100, the system "searches," in the image obtained by the detection system, for the presence of a tire in the environment around the device 100. If no tire is detected, the detection system continues to obtain images until the search of the environment around the device 100 is exhausted. The detection system can determine information about the physical environment that can be used by a control system (which includes, for example, software for planning the movements of the device 100).The control system could be located on the device 100 or it could be in remote communication with the device. In embodiments of the device, one or more 2D or 3D sensors mounted on the device 100 (including, without limitation, navigation sensors) may be integrated to provide a digital model of the physical environment (including, where applicable, the side(s), floor, and ceiling). Using the obtained data, the control system may cause the device 100 to move to navigate it between the target tire engagement positions.

[0054] To properly manage the manipulation of the device 100 that ensures the secure gripping of the target tire (for example, the manipulation of the robot and the positioning of the gripper 108 as shown in Figures 4 and 5), it is necessary to detect the arrangement of the tires arranged in the group of tires and to identify the ideal target tire for gripping. Thus, the detection data refers to a plurality of records representative of the locations of at least one tire or a portion of a tire tracked over time.For example, the sensing data may include one or more of recordings of the positions of a reference point on a portion of the tire (e.g., the sidewall) over time or at defined time intervals; sensor data taken over time; a video stream that has been processed using a computer vision technique; and / or data indicative of the operating state of the device 100 over time. In some cases, the sensing data may include data representative of one or more continuous movements of the gripping device before it stops to take one or more images of the arranged tires. The sensing system of the device 100 is therefore configured to generate the device movement data.In embodiments of the invention, the detection system of the device 100 may also include a motion capture device selected from infrared sensors, ultrasonic sensors, accelerometers, gyroscopes, pressure sensors, and / or other equivalent devices. For example, a motion capture device of the system of the invention may include one or a pair of digital gloves for performing management movements of the device 100 remotely. In these embodiments, the system (and particularly the device 100) learns the movements that achieve the target arrangement of the tires without intervention from an operator during one or more subsequent positioning processes forming part of the chaining method.

[0055] To implement the method of the invention by computer means, the chaining system of the invention comprises a communication network (or "network") which manages the data incoming to the system from various sources (for example, from at least one device 100 and its associated detection system). The communication network incorporates one or more communication servers (or "servers") each comprising one or more processors operatively connected to a memory. The memory is configured to store an application for analyzing data representative of the imaged tires. The one or more processors comprise an analysis application execution module which performs the image processing (or "image processing module"), the one or more processors of which are capable of executing programmed instructions stored in the memory to perform the steps of the chaining method (as described below).Considering the device 100, its initial positioning (and, in applicable cases, the initial orientation of the gripper 108) is determined from data obtained via the acquisition of images of the chaining system and the physical environment in which the chaining system operates. An analysis application execution module of the processor employs an automatic and adaptive repositioning algorithm to find an ideal starting position of the device 100, thus making it possible to execute programmed instructions stored in the memory to carry out the gripping of a first identified target tire to be positioned in a predetermined target location (for example, a trailer 300).The repositioning algorithm allows for continuous improvement across all tire outlets, ensuring that the chaining system (and particularly the device 100) improves from the experience it acquires, particularly in the choice of tires to be released from the first tire identified for chaining.

[0056] The term "processor" (or, alternatively, the term "programmable logic circuit") refers to one or more devices capable of processing and analyzing data and including one or more software programs for processing them (e.g., one or more integrated circuits known to those skilled in the art as being included in a computer, one or more controllers, one or more microcontrollers, one or more microcomputers, one or more programmable logic controllers (or "PLCs"), one or more application-specific integrated circuits, one or more neural networks, and / or one or more other known equivalent programmable circuits). The processor includes one or more software programs for processing the data captured by the detection system of the device 100 (and the corresponding data obtained) as well as one or more software programs for identifying and locating variances and identifying their sources to correct them.

[0057] In the system of the invention, the memory may include both volatile and non-volatile memory devices. The non-volatile memory may include solid-state memories, such as NAND flash memory, keep-alive memory (KAM) for saving various operating variables while the processor is powered off, magnetic and optical storage media, or any other suitable data storage device that retains data when the device 100 is disabled or loses power. The volatile memory may include static and dynamic RAM that stores program instructions and data, including a learning application.In embodiments of the system of the invention, the processor can configure the device (and in particular the gripper 108) on one or more parameters of the identified tire calculated by an image processing module incorporated in the memory of the processor.

[0058] In embodiments of the chaining method of the invention, the processor can configure the chaining system (and in particular the device 100) on one or more parameters of a target tire calculated by an image processing module. In these embodiments, it is understood that one or more means of reinforcement learning could be used. Those skilled in the art in this field will recognize that numerous image processing techniques can be used to choose and determine the parameters of the target tires. Several commercially available image processing systems can be used.

[0059] To implement the method of the invention by computer means, the system of the invention comprises a communication network (or "network") which manages the data incoming to the system from various sources (for example, from at least one device 100 and its associated detection system). The communication network incorporates one or more communication servers (or "servers") each comprising one or more processors operatively connected to a memory. The memory is configured to store an application for analyzing data representative of the positions of the imaged tires. The one or more processors comprise an analysis application execution module which performs the processing of the images, the one or more processors of which are capable of executing programmed instructions stored in the memory to carry out the steps of the method (as described below).

[0060] The input data to the chaining system of the invention may include general information regarding the identified tire. The general information includes stored data regarding the identification of the identified tire (including, without limitation, its production origin, distribution and / or storage, production date, retreading history if applicable and its mounting position and history). The corresponding data could be collected by one or more known devices (e.g., an RFID-type device disposed in or on the identified tire).

[0061] The processor may also refer to a reference (e.g., a table of various tire sizes) to make a final determination of a parameter or parameters of the identified tire. The reference may include known tire parameters corresponding to a plurality of known commercially available tires. For example, after the image processing module calculates one or more identified tire parameters, the processor may compare the calculated parameters with the known parameters stored in the reference. The processor may retrieve the known tire parameters corresponding to the commercially available tires that most closely match the calculated parameters to configure the gripper 108 (and thus place it in a precise position to pick up and position the identified tire).The tire reference may include measurements corresponding to a plurality of commercially available tires. For example, for a tire of size 225 / 50R17, the number "225" identifies the tire's cross-sectional area in millimeters, the number "50" indicates the sidewall aspect ratio, and the measurement "RI 7" represents the rim diameter in inches (being approximately 43.18 centimeters).

[0062] Referring again to Figures 1 to 5, and further to Figures 6 to 19, a detailed description is given by way of example of embodiments of a chaining method (or "method") of the invention allowing the chaining of tires in a target location (e.g., the interior of a trailer 300 of a truck) (see Figure 7). The chaining method is implemented by the system of the invention in any physical environment without prior knowledge of the dimensions of the tires in the group of tires intended for chaining and without prior knowledge of their arrangement. It is therefore understood that the location positions of the tires are generated on the fly to take into account the different dimensions of successive tires depending on the parameters of the target location.

[0063] As used herein, the term “method” or “process” may include one or more steps performed by at least one computer system having one or more processors to execute instructions that perform the steps (e.g., the steps of a positioning algorithm implemented by the system of the invention). Unless otherwise indicated, any sequence of steps is exemplary and does not limit the described methods to any particular sequence.

[0064] In performing the chaining method of the invention, the system of the invention incorporates a combination of vision and machine learning techniques to correctly and quickly reconstruct the observed scene from scattered three-dimensional (or "3D") point clouds, derived from a view of the tire identified for chaining. The system of the invention therefore achieves continuous improvement in the recognition of different tires and their positioning relative to the target location.

[0065] Figure 6 represents a flow diagram of an embodiment of operation of the system of the invention during a chaining process performed by the system. As shown in Figure 6, the "simulator" represents a positioning algorithm implemented by the system of the invention to choose the optimal location of a tire to be chained.

[0066] The positioning algorithm (or "simulator") implemented by the system of the invention can be seen as a physical model seeking to represent the physics associated with chaining tires in a predetermined target location. Like other models, it makes assumptions:

[0067] The workspace and tires are represented in a two-dimensional (“2D”) space. Each tire is represented by a rectangle (where the width of the rectangle represents the width of the tire casing and the height of the rectangle represents its diameter) which will be positioned within a larger rectangle representing the group of tires being chained (see Figure 7).

[0068] The positioning algorithm does not chain the tires, but it looks for positions that favor their chaining. Interaction with a chaining system detection system allows tire chaining to be taken into account in the positioning algorithm.

[0069] It is assumed that there is no slippage between different tires when constructing a stacked tire group. Thus, the phenomenon of tire settling is not taken into account.

[0070] The positioning algorithm is based on rules put in place to position each tire (for example, the orientation of the tires, the concept of “row”, etc.).

[0071] It is clear that this physical model could be refined to improve performance. For example:

[0072] The positioning algorithm could be improved by working in a three-dimensional space, representing each tire by a three-dimensional (3D) shape (e.g., a shape from a CAD model).

[0073] Tire chaining could be taken into account by the positioning algorithm by allowing the nesting of tires represented in 3D.

[0074] Tire slippage and compaction could be incorporated.

[0075] The rules used to position each tire could be replaced by a reinforcement learning algorithm that would seek to learn the most relevant rules on its own. This would increase the chain density achieved.

[0076] Referring again to Figures 6 and 7, in initiating the chaining method of the invention, the method comprises a step of estimating the dimensions of a target location whose tires are intended for chaining.

[0077] For example, in the following description, the trailer 300 of Figure 7 is referenced as the target location that is the subject of this estimation. It is understood that the target location could be chosen from one or more known target locations for carrying out the chaining and transport of tires (including, without limitation, containers, pallets, truck beds, chained trucks, box trucks and their equivalents).

[0078] The estimation step of the chaining method includes a step of scanning the interior of the target location whose tires are intended for chaining. During this step, the chaining system uses its detection system to scan the interior of the target location and to estimate its parameters (i.e., its dimensions). Taking the example of the trailer 300 of Figure 7, during this step, the detection system scans the interior of the trailer to determine its length, width and depth.

[0079] The chaining method further comprises a step of creating the origin of a location-device marker. During this step, to begin chaining tires, it is necessary to know where to begin placing the first tire in the target location from the parameters obtained during the step of scanning the interior of the target location (for example, it is necessary to know where to begin placing in the trailer 300 the tire P200* of a group of tires P200 stored in a box 200) (see Figure 8). As an example, for the remainder of the description, a marker is positioned at the coordinates that correspond to the lower left corner of the trailer 300 (located at the bottom of the trailer 300). This marker will make it possible to control the device 100 so that it positions the tires in the trailer 300. If the marker is placed in another corner, the complete system would operate with some adaptations.

[0080] The chaining method further comprises a step of estimating the dimensions (the diameter and the width) of a target tire to be chained in the predetermined target location (see tire P200* in Figure 8). During the estimation step, the detection system is used to identify the first tire to be taken from among several tires arranged in an unknown arrangement (for example, the group of tires P200). It is understood that a known system for supplying tires to be loaded could be used to bring the tires to the device 100 (for example, a system incorporating one or more belts for transporting the tires to the device 100 and its detection system). The diameter and the width of the tire P200* are shown in an initial positioning in Figure 7 (the initial positioning being defined according to the location-device reference frame created during the step of creating the reference frame origin).

[0081] In one embodiment of a chaining method performed by the chaining system incorporating the device 100 (and / or an equivalent device), the system of the invention performs a learning process from a single or a small number of examples (called "few-shot learning" or "FSL"). Few-shot learning can reduce the data collection effort for data-intensive applications (including image classification and video event detection), helping to alleviate the burden of large-scale supervised data collection (see "One-Shot Learning of Object Categories", Fei-Fei, Li, Fergus, Rob, and Perona Pietro, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, Issue 4, pp. 594-611 (April 2006) (https: / / doi.org / 10.109 / TP AMI.2006.79). This process is performed in a manner disclosed by Applicant's application FR2113035.In one embodiment of the chaining method of the device 100, the system uses few-shot learning to facilitate a storage optimization function aimed at enhancing tire picking. The system of the invention therefore achieves continuous improvement in the selection of tires to be picked.

[0082] In this embodiment of the chaining method performed by the chaining system incorporating the device 100, the few-shot learning process uses an attention mechanism to perform the gripping of target tires. The few-shot learning process benefits from a neural network structure allowing it to focus only on the tires likely to be gripped by recognizing them immediately. This makes the device 100 (and therefore the system of the invention) faster and more robust because it can adapt to all the orientations of the tires that it sees. In addition, the adaptation to the tire arrangements is performed regardless of the configuration of the gripping device of the management system.

[0083] In this embodiment of the chaining method, the few-shot learning process comprises a step of acquiring data corresponding to the arranged tires. In the group of tires intended for chaining in the target location, during this step, the detection system (e.g., an RGB-D camera 400 of the chaining system of the invention and / or the device 100) (see Figure 5) captures an initial image of a group of P200 tires arranged randomly in an unknown location (e.g., a crate 200 as shown in Figures 5 and 8). In this example, several overlapping tires appear in the field of view of the detection system. During this step, a point cloud is constructed from the RGB-D images as described above.

[0084] The few-shot learning process also includes a step of feeding an extraction neural network (or "extraction network") and an attention neural network (or "attention network"). This step includes a step of training the extraction network to segment the scene seen by the detection system. In one embodiment of the method, this step may include a step of segmenting the data based on a plurality of cycles of a repetitive movement of the gripping device during one or more chaining cycles. The segmentation performed during this step differentiates between an object in the image that includes a tire (either a whole tire or a partial tire) and an object in the image that does not include any tire.

[0085] During the feeding step, the extraction network and the attention network are trained by taking a plurality of sample images (obtained during the acquisition step) as training data and a plurality of image object classifications (or "heat maps") as data labels. For example, based on the image object classification, an image may be evaluated to determine whether the image is capable of attracting the attention of the detection system after the image including the tire targeted for capture is returned to the detection system. During this step, the RGB-D camera provides information on the depth of the arranged tires.

[0086] The feeding step includes a step of building an attention mechanism. During this step, the attention network extracts differentiated features among different categories into a target tire detection model (or "template"), so that the model is guided to locate key areas with important features in a segmented image (i.e., an image incorporating the most likely grip tire). The model gives better oversight to the key areas in order to learn differences among easy-to-confuse categories (e.g., the most likely grip tire among tires arranged from a labeled set of images). The accuracy of detecting the target tire in the image is therefore improved to arrive at the choice of the identified target tire for grip.

[0087] In this embodiment of the chaining method, the process further comprises a step of performing a three-dimensional (3D) reconstruction process that serves to provide the geometric information necessary to generate the ideal gripping point by the gripping device (e.g., gripping by a gripper 108 of a device 100). The 3D reconstruction process comprises a reconstruction process fully performed on the basis of the data from the extraction and attention networks. During this step, the orientation, dimensions and location of the identified target tire are reconstructed from this information data. In doing so, the management system (including the device 100) has obtained recognition of the arranged tires (including their orientations and positions) from tire examples during the few-shot learning process.

[0088] During this step, the management system constructs a virtual tire in the shape of a cylinder on the visible surface of the cluster representing a target tire. During this step, the identification of the center of the identified target tire is done to estimate its inner and outer diameter (the inner diameter being represented by twice the rim radius Rj as discussed above with respect to Figure 2). Once the attention network learns the location of the target tire and its orientation, it sends the corresponding coordinates (e.g., X,Y,Z coordinates and tire axis orientation) to the device 100. At this stage of the process, path plans and distance transformations are already done to extract the identified target tire. Thus, it is the management system that has dictated the information expected in return from the detection system (being the RGB-D camera).Accordingly, the device 100 applies the movements necessary to achieve the engagement and disengagement of the target tire at the ideal engagement point.

[0089] The chaining method of the invention further comprises a step of performing a positioning process for each tire to be chained. This step comprises a step of approaching the device 100 towards the target tire identified during the data acquisition step (e.g., the target tire P200* shown in Figure 8). This step further comprises a step of gripping the target tire identified at the ideal gripping point during the data acquisition step (e.g., as calculated during the step of performing the 3D reconstruction process). For the configuration of a device 100 as shown in Figures 4 and 5, during this step, the gripper 108 is managed to engage a sidewall of the target tire (e.g., by extending one or more fingers of the gripper towards a gripping point of the inner boundary Fi of the sidewall F) (see Figure 2).It is understood that the implementation of the method of the invention is not limited by the configuration of the gripper device 108 of the device 100.

[0090] During the step of positioning the target tire identified for pickup, the positioning algorithm implemented by the processor of the chaining system considers only the group of tires being chained. This group is modeled in a two-dimensional (2D) space in the form of a rectangle. The width and height of the rectangle are equal to the width and height of the interior of the target location (for example, the width and height of the trailer 300). The goal of the positioning algorithm is then to position the successive tires in this rectangle. Each tire is also modeled using a rectangle whose width and height are equal to the width and diameter of the tire represented.

[0091] Referring now to Figure 9, Figure 9 represents a tire positioning process carried out during the positioning step. The tire positioning process comprises a step of positioning the first tire that is pickable, this being the “target tire”. This first tire is positioned flat at a place corresponding to the choice of the location-device reference mark (created during the step of creating the origin of the reference mark). For example, this first tire is positioned at the lower left corner of the trailer 300 (this behavior being linked to the choice of the location-device reference mark made previously) (see Figure 7). The tire positioning process further comprises a step of arranging the tires starting from the second tire. During this step, the positioning algorithm uses a notion of row which is a sequence of tires positioned in the same direction.Referring to Figure 10, four (4) types of directions are distinguished, including:

[0092] To the right (represented by arrows A in Figure 10);

[0093] To the left (represented by arrows B in Figure 10);

[0094] Upward following a row to the right (represented by arrows C in Figure 10); and

[0095] Upward following a row to the left (represented by arrow D in Figure 10).

[0096] During the tire stacking step, the chaining system starts by filling a row based on the starting point (which could be “to the right” as represented by arrows A or “to the left” as represented by arrows B) (see Figure 10). Each row type is associated with a set of orientations to be tested for the tire to be positioned (see Figure 11). The chaining system therefore starts by orienting the tire according to the first orientation value to be tested (e.g., the orientation represented by angle 0 or the orientation represented by angle - 0 in Figure H).

[0097] Referring to Figure 12, an example is provided of an implementation of tire stowage in a target location (here, the trailer 300 which is also shown in Figure 7). During the tire stowage step, the positioning algorithm will then apply the following steps to select candidate tire positions:

[0098] A step of first calculating all the positions which bring the tire concerned into contact with the limits of the target location;

[0099] A step then of calculating all the positions which put the tire in contact with the other tires already chained (represented in Figure 12 by the tires Pc);

[0100] A step of preserving, among all the positions calculated during the previous step, positions allowing the tire to have at least two points of contact with the limits of the target location or with the other tires (represented in Figure 12 by the PD tires); and

[0101] A final step of preserving the positions allowing the current row to be completed (represented as an example in Figure 12 by the circled PBON tire which represents the only position allowing the current row to be completed) (here, “to the right”).

[0102] It is understood that the steps of positioning the surrounded PBON tire could be different depending on the parameters of the target location (e.g., the height, width and depth of the trailer 300). In addition, the relative dimensions of the tires in the tire group must be taken into account to select the candidate tire positions (these dimensions are not presumed equal for all tires being chained).

[0103] The tire storage step further includes a step of calculating a stability and chaining criterion. The stability criterion makes it possible to know whether the position of the tire is stable in each position retained at the end of the selection of the candidate positions of the tire. Referring to Figures 13 and 14, the tire P200* represents a tire already positioned, and the tire PBON represents a tire to be positioned (it is understood that the precise positions of the tires P200* and PBON are given as an example and that other positions could be taken). The stability of the position of the new tire depends on the vertical starting from its center of gravity (see arrow E in Figures 13 and 14) and the support zone of the new tire (see zone F shown in Figures 13 and 14). If the vertical does not cross the support zone, the position is unstable (see Figure 14).Otherwise, the position is stable and the stability of the position can be estimated by dividing the length of the support zone F by the diameter of the tire (this estimate being "the stability criterion") (see Figure 13). In this case, the position is preserved if its stability is strictly greater than 0.5.

[0104] It is understood that one or more other embodiments could incorporate slip between tires into the stability criterion to improve the physics of the model.

[0105] The chaining criterion is based on the horizontal signed distance between the center of the last tire installed and the end of the new tire. Only positions that are sufficiently far from the last tire are retained. If several positions are retained, the final position of the new tire will be the one closest to the last tire positioned (to avoid forming gaps, or "holes", that are too large between two successive tires). Conversely, if no position is retained, this means that the end of the row has been reached and the next row must be moved on. The system then moves on to the next row, and it applies the previous steps again to find the best position for the new tire.

[0106] Referring to Figures 15 and 16, an illustration of the chaining criterion is provided which is based on the horizontal signed distance between the center of the last installed tire (see tire 1) and the end of the new tire 2. In Figure 15, the new tire 2 covers the center of the last tire 1, which results in a negative distance and an unfavorable chaining situation (the hollow area of ​​the last tire which could accommodate part of a future installed tire is covered). In Figure 16, the new tire 2 is sufficiently far from the last tire 1 to bring about a favorable chaining situation (which results in a positive distance). For example, another tire 3 installed later can fit into the hollow part of tire 1, and a chaining of the tires is also obtained.

[0107] It is understood that one or more other embodiments could incorporate penetration between tires into the chaining criterion to improve the physics of the model.

[0108] The tire arrangement step further comprises a conversion step. During this step, the 2D center of the tire from the positioning algorithm is converted to the 3D center of the tire in the predetermined target location. Referring to Figure 17, the positioning algorithm indicates the center of the tire in the rear plane of the tire group (represented in Figure 17 by the dotted segments) as well as its orientation (represented in Figure 17 by the angle a). To position the tire in the target location (e.g., trailer 300), it is necessary to know its position in the 3D coordinate system of the target location (defined based on the location-device coordinate system created during the coordinate system origin creation step). For this, the 2D center is shifted forward by an amount equal to the tire radius (e.g., radius Rp ) (see Figure 2).The 3D center of the tire is therefore obtained and coupled with the orientation (angle a) preserved to position the tire in the target location. The chaining method of the invention further comprises a step of scanning the positioned tire after its positioning (performed during the previous step). During this step, the positioning algorithm uses a chaining criterion to choose the position of the tire but it does not chain the tires (it does not manage the penetration of the tires into each other). The chaining of the tires is carried out naturally when the gripper of the device 100 releases the tire. It should therefore be understood that the positioning algorithm calculates a “position before chaining”.During this step, it is necessary to update this “position before chaining” so that it becomes a “position after chaining” which is the actual position of the tire in the predetermined target location (e.g., trailer 300). This is necessary to correctly position the next tires in the target location. Without this update, there would be a drift or a gap between the “positions before chaining” of the positioning algorithm and the actual positions of the tires in the target location.

[0109] To know this actual position, the chaining system incorporates a combination of vision and machine learning techniques to correctly and quickly reconstruct the observed scene from 3D scattered point clouds, derived from a view of a group of stacked tires. The system therefore achieves continuous improvement in tire recognition and their relative positioning along the internal surface of the target location (e.g., trailer 300).

[0110] During the tire scanning stage, the chaining system uses its detection system to scan an area in which the last tire was positioned. During this stage, the attention mechanism (described above) is then used to detect the last tire positioned in the color image.

[0111] In one embodiment, a reference may be created that incorporates coordinates of the centers of the sought tires in images captured by the detection system of the device 100 (e.g., an RGB-type camera). The coordinate reference that is created during this step includes expected images corresponding to the centers distributed in the target location. The images obtained during the scanning step revealing one or more positions of the centers of the tires intended for chaining, train at least one neural network to identify all the expected positions of the centers in the imaged target location. Thus, these image variations serve as input to the neural network whose outputs are the classification of the coordinates of the centers of the tires. In this embodiment of the method, at least a portion of the center reference may be created by one or more skilled persons.

[0112] With the 3D points of this tire retrieved, a cylinder at these 3D points is then adjusted. The 3D points of the cylinder are projected into the rear plane of the tire group to obtain a set of 2D points. In embodiments of the method, during this step, a neural network may be trained to recognize the true coordinates of the centers and to create bounding boxes (or "boxed regions") around the recognized centers. During this training, the coordinates of the bounding box of the recognized center are correlated with the coordinates of the sought centers to calculate displacements between them. The boxed regions and the displacement calculations are transmitted to a neural network (for example, one or more CNNs) to jointly learn the representation of a tire in different perspectives of the images taken by the detection system of the device 100.Thus, the bounding box of these 2D points provides the real center and the real orientation of the tire to be transmitted to the positioning algorithm to update it.

[0113] Referring now to Figures 18 and 19, the chaining method of the invention further comprises a final step of updating the position of the tire. Thanks to the scan carried out during the step of scanning the positioned tire, the actual position of the last positioned tire was estimated (see Figure 18 in which tire 7 is not chained relative to tires 1 to 6). This position includes the 2D center of the tire as well as its orientation. By modifying the values ​​of the 2D center and the orientation with those estimated during the scan, the positioning algorithm is updated with the actual position of the last positioned tire. By displaying the actual positions after this update, it is noted that the tires are indeed chained (see Figure 19 in which tire 7 is indeed chained after updating the positions of tires 1 to 6).

[0114] This update step also allows to check that the actual position estimated in the previous step is consistent with the positions of the other tires (ensuring that the overlap between the last tire (e.g., tire 7 in Figures 18 and 19) and the other tires (e.g., tires 1 to 6 in Figures 18 and 19) is not too high). In addition, this update step allows to check that the actual position estimated in the previous step is consistent with the physical limits of the target location (ensuring that the actual position of the last tire (e.g., tire 7 in Figures 18 and 19) is inside the target location (e.g., the inside of the trailer 300). This step therefore improves the robustness of the complete system by allowing it to detect certain inconsistencies.

[0115] The chaining system of the invention can easily repeat the steps of the chaining method in an order to properly chain the tires of a group in an arrangement that suits the parameters of the target location.

[0116] In all embodiments of a chaining method carried out by the chaining system incorporating at least one device 100, the steps can be carried out simultaneously on all of the tires arranged via parallelization carried out by the processor used.

[0117] In a management installation incorporating the chaining system, the data collected by the sensors can be used in the management of an apparatus which performs the picking up of the tires arranged in the target location. In one embodiment of the management installation, this apparatus comprises at least one device 100 of the type described above and shown in Figures 4 and 5.

[0118] In one embodiment, the sensors may include one or more detection sensors (not shown) that detect the presence of one or more tires based on properties of the target location (e.g., the height and / or width and / or depth of the trailer 300). The detection sensors may be selected from commercially available sensors (e.g., reflector-type sensors). Detection of a first tire at the defined marker in the cargo space may trigger the chaining process performed by the chaining system.

[0119] In one embodiment of the system, the sensors may include one or more detection sensors (not shown) that detect the presence of one or more tires based on properties of the target location (e.g., the height and / or width and / or depth of the trailer 300). The detection sensors may be selected from commercially available sensors (e.g., reflector-type sensors). Detection of a first tire at the defined marker in the cargo space may trigger the chaining process performed by the chaining system.

[0120] In a tire management installation incorporating the chaining system of the invention, a detection system may be used to detect the presence of an arrangement of tires in the field of view of a camera of the detection system (e.g., camera 400), thereby triggering the camera to capture the image of one or more tires. In cases where a portion of the tire is not visible in the image obtained by the camera, an arbitrary point may be placed at a known position relative to the sensor of the detection system (e.g., at a known horizontal distance and a known vertical distance from the position of the sensor).

[0121] The sensors of the target location, the sensors of the device 100 and the vision system of the management facility can therefore provide information about the physical environment that can be used by a control system (which includes, for example, tire chaining planning software, and / or software for controlling the corresponding movements of the device 100). The control system could be in remote communication. In embodiments, one or more sensors mounted on the device 100 (including, without limitation, navigation sensors) can be integrated to form a digital model of the physical environment (including, where applicable, the side(s), floor and ceiling). Using the obtained data, the control system can cause the movement of the device 100 to navigate between the target tire chaining positions according to the parameters of the target location.

[0122] To avoid a complex and slow approach that is difficult to maintain and scale, the disclosed invention combines a positioning algorithm with a 3D vision system of the sensing system to account for the complexities of interactions between different tires (including slippage and chaining). The great advantage of this approach lies in the combination of the positioning algorithm with post-positioning tire scanning steps to account for the complexities of interactions between different tires (including slippage and chaining). It is anticipated that the employed positioning algorithm could operate without additional work on any type of tire of any size.This avoids having to use a positioning algorithm based on a very advanced physical model which would be more complex to use, maintain, and develop and which would present longer calculation times and therefore potentially incompatible with the required cycle times. Consequently, the invention allows the chaining of tires within a constrained space, which presents a complex situation because it is necessary to manage collisions with the internal limits of the space provided for chaining.

[0123] In advance of starting the method of the invention, orders for particular tires could be received by known means (for example, by one or more communication networks integrated in a facility incorporating the chaining system of the invention). The facility allows receiving the orders up to a specific time to ensure the chaining of ordered tires in a "just in time" manner (where the chaining process starts).

[0124] A method of the invention may be carried out by PLC control and may include pre-programming of management information. For example, a process setting may be associated with the parameters of the target location being warned and / or the properties of the tires intended for chaining. The system of the invention (and / or a facility incorporating this system) may easily repeat one or more steps of the method in a determined order to properly supply ordered tires to obtain a desired tire chaining.

[0125] The system of the invention (and / or a facility incorporating this system) may include pre-programming of management information. For example, a process setting may be associated with the parameters of the typical physical environments in which the system operates. In embodiments of the invention, the system (and / or a facility incorporating this system) may receive voice commands or other audio data representing, for example, a step or stop of the device 100 and / or a chaining of one or more tires in an identified tire group. The request may include a request for the current state of a chaining cycle. A generated response may be represented audibly, visually, tactilely (e.g., using a haptic interface), and / or virtually and / or augmentedly. This response, associated with the corresponding data, may be recorded in a neural network.

[0126] For all embodiments of the system, a monitoring system could be implemented. At least part of the monitoring system may be provided in a portable device such as a mobile network device (e.g., a mobile phone, a laptop, one or more network-connected wearable devices (including “augmented reality” and / or “virtual reality” devices), network-connected wearables, and / or any combinations and / or equivalents). It is envisaged that detection and comparison steps may be performed iteratively.

[0127] The terms "at least one" and "one or more" are used interchangeably. Ranges that are presented as "between a and b" encompass the values ​​"a" and "b".

[0128] Although particular embodiments of the disclosed apparatus have been illustrated and described, it will be understood that various changes, additions, and modifications may be practiced without departing from the spirit and scope of the present disclosure. Accordingly, no limitations should be imposed on the scope of the disclosed invention except those set forth in the appended claims.

Claims

Claims 1. A method for automatically chaining tires from a group of tires in an unknown arrangement and intended for chaining in a predetermined target location, the chaining method implemented by at least one processor comprising an image processing module which employs a positioning algorithm to choose the optimal location of a target tire to be chained, characterized in that the chaining method comprises the following steps: a step of providing a chaining system of which the processor is a part, wherein the chaining system comprises at least one gripping device (100) which performs the gripping of a target tire of the group of tires; a step of estimating the dimensions of a target location of which the tires of the group of tires are intended for chaining, this step comprising a step of scanning the target location to determine its parameters;a step of creating the origin of a location-device reference mark during which the chaining system creates a reference mark to know the positioning coordinates of a first tire in the target location; a step of estimating the dimensions of a target tire to be chained in the target location, during which the device (100) identifies the first tire to be taken from the group of tires; a positioning step for each tire to be chained, this step comprising the following steps: a step of carrying out a tire positioning process which is carried out for each tire to be chained; a step of approaching the device (100) towards the identified target tire; and a step of taking the identified target tire; a step of scanning the tire after its positioning carried out during the positioning step;and a final step of updating the position of the tire, during which the actual position of the last positioned tire, estimated during the step of scanning the positioned tire, includes the 2D center of the positioned tire and its orientation, which makes it possible to update the positioning algorithm with the actual position of the last positioned tire; such that the update makes it possible to check that the estimated actual position is consistent with the physical limits of the target location; and such that the chaining of the tires is carried out when the gripping device (100) releases the tire.

2. The chaining method of claim 1, wherein, during the step of performing a positioning process for each tire to be chained: the group of tires is modeled in a two-dimensional (2D) space as a rectangle, wherein the width and height of the rectangle are equal to the width and height of the interior of the target location; and each tire of the group of tires is modeled using a rectangle whose width and height are equal to the width and diameter of the tire represented.

3. The chaining method of claim 1 or claim 2, wherein the step of estimating the dimensions of a target tire to be chained comprises a few-shot learning process comprising the following steps: a step of acquiring data corresponding to the arranged tires, during which a detection system of the gripping device (100) captures an initial image of the randomly arranged tires (P200); a step of feeding an extraction neural network and an attention neural network, during which the two neural networks are trained by taking a plurality of sample images obtained during the data acquisition step as training data and a plurality of image object classifications as data labels;a step of performing a three-dimensional (3D) reconstruction process entirely carried out on the basis of the data of the extraction neural network and the attention neural network, during which coordinates corresponding to the location of an identified target tire and its orientation are reconstructed from these data, so that this reconstruction serves to provide the geometric information necessary to generate an ideal grip point of the identified target tire; a step of approaching the gripping device (100) towards the identified target tire, during which the attention neural network sends the coordinates to the gripping device; corresponding to the location of the identified target tire and its orientation; and a step of outputting the identified target tire from the tire arrangement to put it in the target location.

4. The chaining method of claim 3, wherein the step of feeding the extraction neural network and the attention neural network comprises the following steps: a step of training the extraction neural network to segment a scene viewed by the detection system of the management system (100); and a step of constructing the attention mechanism, during which the attention neural network extracts features differentiated among different categories in a target tire detection model, so that the model is guided to locate key areas in a segmented image; wherein the step of training the extraction neural network comprises a step of segmenting data based on a plurality of cycles of a repetitive movement of the gripping device during one or more chaining cycles.

5. The chaining method of claim 3 or claim 4, wherein, during the step of performing the 3D reconstruction process, the orientation, dimensions and location of the identified target tire are reconstructed from the data of the extraction neural network and the attention neural network.

6. The chaining method of claim 4 or claim 5, wherein: the step of acquiring the data of the few-shot learning process comprises a step of constructing a point cloud from the RGB-D images; and the step of approaching the gripping device (100) comprises a step of gripping the identified target tire at the ideal gripping point calculated during the step of performing the 3D reconstruction process.

7. The chaining method of any one of the preceding claims, wherein the tire positioning process comprises the following steps: a step of positioning the first tire which is takeable from the group of tires of which this first tire is positioned flat at a corresponding place the choice of the location-device reference frame created during the step of creating the origin of the reference frame; and a step of arranging the tires starting from the second tire, during which the positioning algorithm uses a notion of row which is a sequence of tires positioned in the same direction, and in which each type of row is associated with a set of orientations to be tested for the tire to be positioned.

8. The chaining method of claim 7, wherein, during the step of arranging the tires, the positioning algorithm applies the following steps to select the candidate positions of the tire: a step of first calculating all the positions which put the tire in contact with the limits of the target location; a step of then calculating all the positions which put the tire in contact with the other tires already chained (Pc); a step of preserving, among all the positions calculated during the previous step, the positions allowing the tire to have at least two points of contact with the limits of the group of tires or with the other tires (PD); and a last step of preserving the positions allowing the current row to be completed (PBON) which represents the only position allowing the current row to be completed.

9. The chaining method of claim 7 or claim 8, wherein the tire arranging step further comprises: a step of calculating a stability and chaining criterion for determining whether the tire position is stable at each position retained at the end of selecting the candidate tire positions; and a conversion step during which the 2D center of the tire from the positioning algorithm is converted to the 3D center of the tire in the target location; such that the positioning algorithm indicates the center of the tire in the rear plane of the tire group and its orientation (a) to position the tire in the target location so as to obtain the 3D center of the tire and couple it with the orientation (a).

10. The chaining method of claim 9, wherein, during the step of calculating a stability and chaining criterion: the stability of the position of the new tire depends on the vertical (E) starting from its center of gravity and the support zone (F) of the new tire; and the chaining criterion is based on a horizontal signed distance between the center of the last tire installed and the end of the new tire.

11. The chaining method of any one of claims 4 to 10, wherein, during the tire scanning step performed during the positioning step: the chaining system scans an area in which the last tire was positioned; and an attention mechanism is then used to detect the last positioned tire.

12. A system for automatically chaining tires from a group of tires in an unknown arrangement and for which a predetermined target location is to be achieved, characterized in that the chaining system comprises: at least one gripping device (100) comprising a robot having a gripping device (104) supported by an elongated arm (106) pivoting and extending from the elongated arm (106) to a free end (104a) where a gripper (108) is arranged along a common longitudinal axis; a detection system for collecting information on the physical environment around each gripping device (100), the detection system comprising one or more sensors configured to perform two-dimensional (2D) and / or three-dimensional (3D) image detection;and a communication network that manages incoming data to the chaining system from at least one socket device (100), the communication network comprising at least one communication server for executing programmed instructions stored in a memory of one or more processors of the chaining system that employs a positioning algorithm to implement the chaining method of any one of claims 1 to 11; such that each cleaning device (100) is configured on one or more parameters of at least one identified tire calculated by an image processing module incorporated in the memory of the processor to set the cleaning device (100) in motion so that the gripping device (104) can carry out the gripping of a target tire during the chaining process carried out by the chaining system.

13. The chaining system of claim 12, wherein the detection system comprises at least one RGB-D type camera (200) attached to at least one of the gripping device (104), the elongated arm (106) and the gripper (108) of the cleaning device (100).

14. The chaining system of claim 12 or claim 13, wherein the predetermined target location comprises at least one location selected from one or more containers, one or more trucks, one or more trailers, one or more warehouses, one or more distribution centers, one or more conveyors, one or more pallets and one or more storage means.

Images