Method and system for determining condition of component of remotely operated vehicle

Abstract

The invention relates to a method for determining a condition of a component of a remote operating vehicle (501) operating on a track system (108) of an automated storage and retrieval system (1) for cargo holders (106). The method comprises: selecting a component of the remotely operated vehicle (501); recording an image of the component; identifying the component based on the recorded image; and determining the identified condition of the component using the recorded image. The invention also relates to a system (510) for determining the condition of a component and to an automatic storage and retrieval system (1) comprising said system (510).

Description

Technical Field

[0001] The present invention relates to a method for determining the condition of a component of a remotely operated vehicle, a system for determining the condition of a component, and an automatic storage and retrieval system including the system. Background Art

[0002] Figure 1 A prior art automatic storage and retrieval system 1 having a frame structure 100 is disclosed, and Figure 2 , Figure 3a to Figure 3b Three different prior art container handling vehicles 201, 301, 401 suitable for operating on such a system 1 are disclosed.

[0003] The frame structure 100 comprises upright members 102 and a storage volume comprising storage columns 105 arranged in rows between the upright members 102. In these storage columns 105, storage containers 106 (also called boxes) are stacked one on top of the other to form container stacks 107. The members 102 may typically be made of metal, such as extruded aluminum profiles.

[0004] The frame structure 100 of the automated storage and retrieval system 1 includes a rail system 108 arranged across the top of the frame structure 100, and a plurality of container handling vehicles 301, 401 can run on the rail system 108 to lift and lower storage containers 106 from and into the storage rows 105, and also to transport storage containers 106 over the storage rows 105. The rail system 108 includes: a first set of parallel rails 110 arranged to guide the movement of the container handling vehicles 301, 401 across the top of the frame structure 100 in a first direction X; and a second set of parallel rails 111 arranged perpendicular to the first set of rails 110 to guide the movement of the container handling vehicles 301, 401 in a second direction Y perpendicular to the first direction X. The containers 106 stored in the rows 105 are accessed by the container handling vehicles 301, 401 through access openings in the rail system 108. The container handling vehicles 301 , 401 can be moved laterally above the storage row 105 , ie in a plane parallel to the horizontal XY plane.

[0005] The upright members 102 of the frame structure 100 may be used to guide the storage containers during raising and lowering of the containers from the row 105 and into the row. The stack 107 of containers 106 is typically self-supporting.

[0006] Each prior art container handling vehicle 201, 301, 401 includes a vehicle body 201a, 301a, 401a and a first set of wheels and a second set of wheels 201b, 201c, 301b, 301c, 401b, 401c, which respectively enable the container handling vehicle 201, 301, 401 to move laterally in the X direction and the Y direction. Figures 2 to 3b , the two wheels in each set are fully visible. The first set of wheels 201b, 301b, 401b is arranged to engage with two adjacent tracks of the first set of tracks 110, and the second set of wheels 201c, 301c, 401c is arranged to engage with two adjacent tracks of the second set of tracks 111. At least one set of wheels 201b, 201c, 301b, 301c, 401b, 401c can be raised and lowered so that the first set of wheels 201b, 301b, 401b and / or the second set of wheels 201c, 301c, 401c can engage with the corresponding set of tracks 110, 111 at any time.

[0007] Each prior art container handling vehicle 201, 301, 401 further comprises a lifting device 304, 404 (in Figure 3a to Figure 3b ), such as raising the storage container 106 from the storage row 105 and lowering the storage container 106 into the storage row 105. The lifting device 304, 404 includes one or more clamping / engaging devices adapted to engage the storage container 106, and these clamping / engaging devices can be lowered from the vehicle 201, 301, 401 so that the position of the clamping / engaging device relative to the vehicle 201, 301, 401 can be in a third direction Z orthogonal to the first direction X and the second direction Y (e.g., in Figure 1 Some parts of the clamping device of the container handling vehicle 301, 401 are adjusted. Figure 3a and Figure 3b The container handling device 201 has a gripping device located at Figure 2 The vehicle body 201a is located within the vehicle body 201a.

[0008] Conventionally, and also for the purposes of the present application, Z=1 identifies the uppermost level of storage containers that may be used below rails 110, 111, i.e., the level directly below rail system 108, Z=2 identifies the second level below rail system 108, Z=3 identifies the third level, etc. Figure 1 In the exemplary prior art disclosed in , Z=8 identifies the lowest bottom layer of the storage container. Similarly, X=1...n and Y=1...n identify the position of each storage column 105 in the horizontal plane. Thus, for example, and using Figure 1 The Cartesian coordinate system X, Y, Z shown in Figure 1 The storage container denoted as 106' in FIG. 1 can be said to occupy storage position X=18, Y=1, Z=6. The container handling vehicles 201, 301, 401 can be said to be traveling in level Z=0, and each storage column 105 can be identified by its X and Y coordinates. Thus, Figure 1 The storage containers shown extending above the rail system 108 can also be referred to as being arranged in level Z=0.

[0009] The storage volume of the frame structure 100 is generally referred to as a grid 104, wherein the possible storage positions within the grid are referred to as storage units. Each storage column can be identified by a position in the X-direction and the Y-direction, and each storage unit can be identified by a container number in the X-direction, the Y-direction, and the Z-direction.

[0010] Each prior art container handling vehicle 201, 301, 401 includes a storage compartment or space for receiving and loading storage containers 106 when transporting storage containers 106 across the track system 108. The storage space may include a cavity disposed within the vehicle body 201a, such as Figure 2 and Figure 3b As shown in, and as described in, for example, WO2015 / 193278A1 and WO2019 / 206487A1, the contents of which are incorporated herein by reference.

[0011] Figure 3a An alternative construction of a container handling vehicle 301 having a cantilever structure is shown. Such a vehicle is described in detail in, for example, N0317366, the contents of which are also incorporated herein by reference.

[0012] Figure 2 The chamber container handling vehicle 201 shown in can have a footprint covering an area in the X-direction and the Y-direction, the size of which is generally equal to the lateral extent of the storage row 105, for example as described in WO2015 / 193278A1, the contents of which are incorporated herein by reference. The term "lateral" as used herein can mean "horizontally".

[0013] Alternatively, the footprint of the cavity container handling vehicle 401 may be larger than the lateral area defined by the storage row 105, such as Figure 3b As shown in and as disclosed in WO2014 / 090684A1 or WO2019 / 206487A1.

[0014] The track system 108 typically includes a track with grooves in which the wheels of the vehicle travel. Alternatively, the track may include an upwardly extending element, wherein the wheels of the vehicle include flanges to prevent derailment. These grooves and upwardly extending elements are collectively referred to as guide rails. Each track may include one guide rail, or each track may include two parallel guide rails; in other track systems 108, each track in one direction may include one guide rail, and each track in the other perpendicular direction may include two guide rails. The track system may also include a double guide rail track in one direction of the X direction or the Y direction and a single guide rail track in the other direction of the X direction or the Y direction. The double guide rail track may include two track members fastened together, each track member having a guide rail.

[0015] WO 2018 / 146304 A1 (the contents of which are incorporated herein by reference) shows a typical construction of a track system 108 comprising tracks and parallel guides in both the X-direction and the Y-direction.

[0016] In the frame structure 100, most of the rows 105 are storage rows 105, i.e. rows 105 in which storage containers 106 are stored in the form of stacks 107. However, some of the rows 105 may have other purposes. Figure 1 In the embodiment of the present invention, the columns 119 and 120 are such special columns for unloading and / or picking up storage containers 106 by using container handling vehicles 201, 301, 401, so that they can be transported to access stations (not shown), where the storage containers 106 can be accessed from the outside of the frame structure 100, or the storage containers can be transferred out of or into the frame structure 100. In the art, such positions are generally referred to as "ports", and the columns where the ports are located can be referred to as "port columns" 119, 120. The transportation to the access station can be in any direction, i.e., horizontal, inclined and / or vertical. For example, the storage container 106 can be placed in a random or special column 105 within the frame structure 100, and then picked up by any container handling vehicle and transported to the port columns 119, 120 for further transportation to the access station. The transportation from the port to the access station may need to move along multiple different directions such as by a distribution vehicle, a trolley or other transportation route. Note that the term "inclined" refers to the transport of storage container 106 with a general transport orientation somewhere between horizontal and vertical.

[0017] exist Figure 1 In the embodiment, the first port column 119 can be, for example, a dedicated unloading port column, wherein the container handling vehicles 201, 301 can unload the transported storage containers 106 to the access station or transfer station, and the second port column 120 can be a dedicated picking port column, wherein the container handling vehicles 201, 301, 401 can pick up the storage containers 106 that have been transported from the access station or transfer station.

[0018] The access station may typically be a pick-up station or staging station where product items are removed from or positioned in the storage container 106. In the pick-up or staging station, the storage container 106 is typically not removed from the automated storage and retrieval system 1, but rather returned to the frame structure 100 after access. The port may also be used to transfer the storage container to another storage facility (e.g., to another frame structure or to another automated storage and retrieval system), to a transport vehicle (e.g., a train or truck), or to a production facility.

[0019] A conveyor system including conveyors is typically employed to transport storage containers between the port rows 119, 120 and the access station.

[0020] If the port rows 119, 120 and the access station are located at different heights, the conveyor system may include a lifting device with a vertical component for vertically transporting the storage containers 106 between the port rows 119, 120 and the access station.

[0021] The conveyor system may be arranged to transfer storage containers 106 between different frame structures, for example as described in WO 2014 / 075937 A1, the contents of which are incorporated herein by reference.

[0022] When accessing data stored in Figure 11, a storage container 106 in one of the columns 105 disclosed in the disclosure is located, a container handling vehicle 201, 301, 401 is instructed to take out the target storage container 106 from its position and transport the storage container to the unloading port column 119. This operation involves moving the container handling vehicle 201, 301 to a position above the storage column 105 where the target storage container 106 is located, taking out the storage container 106 from the storage column 105 using a lifting device (not shown) of the container handling vehicle 201, 301, 401, and transporting the storage container 106 to the unloading port column 119. If the target storage container 106 is located deep within the stack 107, that is, one or more other storage containers 106 are positioned above the target storage container 106, the operation also involves temporarily moving the storage container positioned above before lifting the target storage container 106 from the storage column 105. This step, sometimes referred to in the art as "digging," can be performed using the same container handling vehicle that is subsequently used to transport the target storage container to the unloading port row 119, or using one or more other cooperating container handling vehicles. Alternatively or in addition, the automated storage and retrieval system 1 can have container handling vehicles 201, 301, 401 that are dedicated to the task of temporarily removing storage containers 106 from the storage row 105. Once the target storage container 106 has been removed from the storage row 105, the temporarily removed storage container 106 can be relocated to the original storage row 105. Alternatively, however, the removed storage container 106 can be relocated to other storage rows 105.

[0023] When a storage container 106 is to be stored in one row 105, one container handling vehicle 201, 301, 401 is instructed to pick up a storage container 106 from the pick-up port row 120 and transport the storage container to a position above the storage row 105 in which it is to be stored. After the storage container 106 positioned at or above the target position within the stack 107 has been removed, the container handling vehicle 201, 301, 401 positions the storage container 106 at the desired position. Then, the semi-removed storage container 106 can be lowered back into the storage row 105 or repositioned to another storage row 105.

[0024] In order to monitor and control the automatic storage and retrieval system 1, for example, to monitor and control the position of each storage container 106 in the frame structure 100, the content of each storage container 106, and the movement of the container handling vehicles 201, 301, 401, so that the desired storage container 106 can be delivered to the desired location at the desired time without the container handling vehicles 201, 301, 401 colliding with each other, the automatic storage and retrieval system 1 includes a control system (such as Figure 1 As shown), the control system is typically computerized and typically includes a database for tracking the storage container 106.

[0025] In such Figure 1 In the context of the storage system shown, different kinds of system maintenance can be achieved. Historically, the most common of these has been corrective maintenance, which involves replacing components sequentially as they fail. Corrective maintenance ensures that components are used until the end of their life, but has the disadvantage of causing significant system downtime due to unplanned component replacements.

[0026] Alternatively, the system can be maintained preventively. Within the framework of a preventive maintenance program, components are replaced according to a predetermined schedule before the end of their life. Thus, component failures can be completely eliminated. Preventive maintenance minimizes unexpected failures and all the disadvantages associated with them, but incurs increased costs.

[0027] Predictive maintenance combines the best aspects of corrective and preventive maintenance by maximizing the useful life of components while reducing the frequency of failures. To achieve this, maintenance work is performed at the optimal moment. To accurately determine the most appropriate moment to perform maintenance, the Weibull distribution can be used. In maintenance engineering, the Weibull distribution is a well-known method for modeling component life distributions and can be used in real-life systems such as Figure 1 storage system).

[0028] In a related context, WO2021 / 198093 discloses a conventional storage system, which also includes a service station for performing maintenance on storage system components (e.g., robots). The system includes a central computer system and a service status manager configured to retrieve condition-based information associated with the components of the storage system, and based on the retrieved condition-based information, create a service policy and send the service policy to the service station.

[0029] Unexpected events may occur in the context of system maintenance, such as the end customer replacing a part with a new one from an unknown source without notifying the system manufacturer, or the end customer performing maintenance on his own initiative. As a result, the manufacturer's maintenance records become incomplete. As a result, the information in the originally planned maintenance schedule becomes inaccurate.

[0030] Improper maintenance of components may lead to component damage. In order to detect such damage in a timely manner, the service operator must have considerable skills and experience. In a related context, it is beneficial to uniformly perform the damage detection process across the entire service operator team.

[0031] It would be desirable to provide a solution that solves or at least alleviates this and other problems pertaining to the prior art. Summary of the invention

[0032] The invention is set forth and characterized in the independent claim, while the dependent claims describe further characteristics of the invention.

[0033] A first aspect of the invention relates to a method for determining a condition of a component of a remotely operated vehicle operating on a track system of an automated storage and retrieval system for cargo holders, the method comprising:

[0034] - Select components for remotely operated vehicles,

[0035] - record an image of the part,

[0036] - identify the component based on the recorded image,

[0037] - Determining the condition of the identified component using the recorded image.

[0038] The solution according to the above method helps to achieve consistent component inspection. More specifically, operator involvement can be greatly reduced or even completely eliminated, thereby achieving a highly standardized component inspection procedure.

[0039] In addition, the method helps to more accurately determine the condition of vehicle components and more accurately schedule component maintenance. As a result, component diagnostics are improved.

[0040] In some implementations, the method may utilize machine learning to process the large amounts of data generated and identify patterns and / or potentially extract deep information that less sophisticated techniques would not be able to identify.

[0041] In addition, the method can also be used to troubleshoot a faulty remotely operated vehicle. In a related context, the method can be used to obtain confirmation that a particular remotely operated vehicle is healthy and operational. This is particularly useful in the event that a general error code is generated that can be attributed to any of the remotely operated vehicle / cargo holder / storage grid.

[0042] A second aspect of the present invention is directed to a system for determining the condition of a component of a remotely operated vehicle operating on a track system of an automated storage and retrieval system for cargo holders.

[0043] The advantages discussed above in connection with the method may even be associated with the corresponding system and are not discussed further, but are equally applicable.

[0044] For the purposes of this application, the term "container handling vehicle" used in the "Background" section of this application and the term "remotely operated vehicle" used in the "Summary" and "Specifications" sections define a robotic wheeled vehicle that is provided with a lifting device and operates on a rail system of a frame structure that is part of an automated storage and retrieval system.

[0045] Similarly, the term "storage container" used in the "Background" section of this application and the term "cargo holder" used in the "Summary" and "Detailed Description" sections all define a container for storing items that can be engaged with the lifting device of the robotic wheeled vehicle. In this context, the cargo holder can be a box, a tote, a pallet, a tray, or the like. Different types of cargo holders can be used in the same automated storage and retrieval system.

[0046] Relative terms "upper", "lower", "below", "above", "higher" etc. are to be understood in their normal meaning and as seen in a Cartesian coordinate system. When referring to a track system, "upper" or "above" is to be understood as a position closer to the surface of the track system (relative to another component), and conversely, the terms "lower" or "below" are to be understood as a position further away from the track system (relative to another component).

[0047] Finally, the remotely operated vehicle of the present invention is typically operated on a track system arranged to span the top of an automated storage and retrieval system for cargo holders. This configuration is also Figure 1 However, it is also conceivable that the remotely operated vehicle operates on a rail system arranged alongside the storage grid, so that the cargo holders are inserted / extracted laterally from the storage grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The following drawings are attached to facilitate understanding of the present invention. The drawings illustrate embodiments of the present invention, which will now be described by way of example only, in which:

[0049] Figure 1 It is a three-dimensional diagram of the frame structure of an automatic storage and retrieval system belonging to the prior art.

[0050] Figure 2 is a perspective view of a prior art container handling vehicle having a centrally disposed cavity for carrying a storage container therein.

[0051] Figure 3a is a perspective view of a prior art container handling vehicle having a cantilever arm for carrying a storage container underneath.

[0052] Figure 3b is a perspective view of a prior art container handling vehicle viewed from below, the container handling vehicle having an internally disposed cavity for carrying a storage container therein.

[0053] Figure 4a is a side perspective view of a system for determining a condition of a component of a remotely operated vehicle.

[0054] Figure 4b yes Figure 4a Top view of the system shown. DETAILED DESCRIPTION

[0055] In the following, embodiments of the present invention will be discussed in more detail with reference to the accompanying drawings. It should be understood, however, that the drawings are not intended to limit the present invention to the subject matter depicted in the drawings.

[0056] The frame structure 100 of the automatic storage and retrieval system 1 is shown in FIG. Figure 3b The prior art frame structure 100 described herein is constructed by using a plurality of upright members 102 , wherein the frame structure 100 further includes a first upper rail system 108 in the X-direction and the Y-direction.

[0057] The frame structure 100 further comprises storage compartments in the form of storage rows 105 arranged between the components 102 , wherein storage containers 106 can be stacked in the form of stacks 107 within the storage rows 105 .

[0058] The frame structure 100 may have any size. In particular, it should be understood that the frame structure may be larger than Figure 1 The structures disclosed in 100 are significantly wider and / or longer and / or deeper. For example, the horizontal extent of the frame structure 100 may exceed 700×700 rows and its storage depth may exceed twelve containers.

[0059] Figure 4a 5 is a perspective side view of a system 510 for determining the condition of a component of a remotely operated vehicle 501. The vehicle 501 is positioned adjacent to an inspection device 515. Once the vehicle 501 is in the correct position, a method for determining the condition of a component of the remotely operated vehicle 501 can be implemented. The method includes selecting a component of the remotely operated vehicle 501 and recording an image of the component. Subsequently, the component is identified based on the recorded image, whereby the condition of the identified component is determined using the recorded image. Here, identification of a component will be interpreted as identifying a specific component that is part of the vehicle 501 being inspected. Identification of a component will enable tracking of the component composition of each vehicle 501. This can be used to issue specific, vehicle-related maintenance tasks or adjust the time interval between two consecutive vehicle inspections and / or component replacements. In an embodiment, identifying a component based on a recorded image may include reading ID data associated with the component. Here, the identification process may be based on optical character recognition (OCR). Figure 4b yes Figure 4a Top view of the system shown.

[0060] An exemplary sequence of a planned maintenance task begins with positioning the vehicle 501 near the inspection device 515 and optionally removing a cover so that different vehicle components become visible. The operator then opens a software application on the handheld device (discussed further below) and connects the vehicle 501 to be inspected to the system 510. The system 510 then prepares the correct maintenance procedure for the components of the submodules of the vehicle (moving, lifting, changing tracks, charging, etc.) and retrieves the appropriate operating data or batch data. The system 510 then guides the operator through the steps of the maintenance procedure, during which image recording is performed. If a deviation is detected during the maintenance procedure, a more specific camera view is requested based on, for example, operating data or batch data. Where appropriate, the defective component is replaced and the component maintenance record is updated appropriately. In this case, a new image of the component is recorded and stored in the system for future reference.

[0061] The component information obtained will also be used to improve the manufacturing quality of individual components and / or to generate realistic estimates of operating costs during the planning phase of new automated storage and retrieval systems (e.g. Figure 1 as shown).

[0062] The components may be wheels of the vehicle 501, a lifting device for vertically transporting a cargo holder (304, 404; in Figure 3a to Figure 3b ), any of a gripper element, a gear box, a motor belt, and a cargo retainer contact sensor.

[0063] Typically, the component condition determined is the remaining useful life (RUL). Alternatively, the component condition determined is the probability of component failure within a given time window.

[0064] Thus, the system 510 can help the operator estimate the degradation rate / RUL of each individual component. By utilizing the operational data, the change in condition between two consecutive inspections is determined. By also taking into account the normal degradation rate of that particular component, a RUL estimate can be calculated. This information can be used to propose or postpone replacement of individual components.

[0065] The solution according to the above method helps to achieve consistent component inspection. More specifically, operator involvement can be greatly reduced or even completely eliminated, thereby achieving a highly standardized component inspection procedure. In addition, the method helps to more accurately determine the condition of vehicle components and more accurately schedule component maintenance. As a result, component diagnosis is improved.

[0066] The method may also be used to troubleshoot a faulty remotely operated vehicle 501. In a related context, the method may be used to obtain confirmation that a particular remotely operated vehicle 501 is healthy and operational. This is particularly useful in the event that the system 1 generates a general error code attributable to any of the remotely operated vehicle / cargo holder / storage grid.

[0067] In the context of the above, the troubleshooting sequence typically begins with removing the vehicle 501 from the grid in response to an error code. Subsequently, the operator will activate the inspection device 515 and enter the vehicle ID. The device 515 then retrieves and analyzes different types of data (e.g., operational and / or batch data) and returns information about which parts of the vehicle 501 may be damaged and should be inspected more thoroughly. Subsequently, an image of the part is recorded. The information contained in the image can also be used to confirm the identity of the specific vehicle 501. After analysis, the inspection device will determine whether to release the vehicle 501 to the grid, or whether the vehicle is indeed faulty and should remain off the grid.

[0068] If necessary, before selecting components of the remotely operated vehicle 501, for example, Figure 4a to Figure 4b The system 510 shown in FIG. 5 identifies the remotely operated vehicle 501. The identification of the remotely operated vehicle 501 generally includes recording an image of the vehicle 501, specifically recording a section of the vehicle 501 that provides vehicle ID information.

[0069] In an embodiment that can be easily combined with the previously discussed embodiments, further information about vehicle components may be collected by sensors arranged at the remotely operated vehicle 501. By way of example, the sensor may be a proximity sensor or a vibration sensor.

[0070] In an embodiment, determining the condition of the identified component includes utilizing operational data associated with the component. For example, the operational data may be data collected during a testing phase of the vehicle 501 or data collected while the vehicle 501 is in operation.

[0071] In one embodiment, based on the operational data, Figure 4a The system 510 shown in will guide the operator to focus on the area of ​​concern. For example, if the system 510 finds a lifting problem with a specific vehicle 501 based on the operational data, the system 510 will request a detailed view of the specific components of the vehicle 501.

[0072] If a damaged vehicle component is subsequently identified, the system 510 will provide this information to the operator.

[0073] In an embodiment, if an operator discovers a defective part during a manual vehicle inspection that has been overlooked by the inspection device 515, the operator will enter this information into the system 510, causing the system to be updated.

[0074] In one embodiment, the collected image data about the component is registered on a memory unit (not shown) of the remotely operated vehicle 501. Advantageously, the registered data can be transferred from the memory unit to the central processing unit while the remotely operated vehicle 501 is being charged. The data transfer can occur wirelessly using a suitable protocol or by tethering the vehicle 501 to an external device. In a preferred embodiment, the data transfer is cumulative, and only new data generated between two consecutive data transfers is transferred.

[0075] In another embodiment, determining the condition of the identified component includes comparing the newly recorded image of the component with a previously recorded image of the component. Typically, the relevance of the data is improved by cleaning the previously accumulated image data. Data cleaning is the process of identifying and replacing incomplete or inaccurate records from a data set.

[0076] In yet another embodiment, determining the condition of the identified component includes utilizing batch information associated with the component. Thus, the time interval between two consecutive inspections and / or component replacements can be adjusted. More specifically, certain batches may have known problems and components belonging to these batches should be replaced immediately. On the other hand, components belonging to batches with excellent quality may have an extended service life.

[0077] In another embodiment, determining the condition of the identified component includes considering environmental conditions, such as air temperature and / or air humidity associated with the automated storage and retrieval system 1 .

[0078] In a related context, information about the condition of a component can be used to reassign a vehicle 501 having a component that appears damaged from a data set value to a task commensurate with the current condition of the component, which task typically requires less effort / accuracy / speed than the normally assigned task. For example, a vehicle 501 that has been identified as having a component that may appear damaged may be instructed to collect only empty cargo holders 106, whereas previously the vehicle may have collected all types of cargo holders (including heavily / unevenly loaded cargo holders). Alternatively and as discussed above, information about the condition of a component can be used to replace the component and then appropriately update the maintenance record for the particular vehicle 501.

[0079] In an embodiment, the time for the next vehicle maintenance can be set based on the determined vehicle component condition. Likewise, the maintenance record of the specific vehicle 501 can be updated afterwards.

[0080] With reference to the embodiments discussed above, if the condition of the identified component is not satisfactory, an image of a selected section of the identified component may be recorded. In other words, it may be considered to zoom in on a specific section of the component. The section of the component is selected based on at least one of the operational data associated with the component, the batch information associated with the component, and the environmental conditions associated with the automatic storage and retrieval system 1.

[0081] In one embodiment (not shown), an image of the vehicle component is recorded by means of a camera of a handheld device associated with the operator. In the same context, information about the identified component is presented to the operator on a display of the handheld device.

[0082] In another embodiment (in Figure 4a to Figure 4b ), an image of a vehicle component is recorded by means of at least one fixed position camera 505. Still referring to Figure 4a to Figure 4b , the remotely operated vehicle 501 is arranged to rotate before recording the image of the component. In an alternative embodiment (not shown), the image of the component is recorded by means of at least one camera provided on a movable robotic arm.

[0083] As stated above, Figure 4a is a perspective side view of a system 510 for determining a condition of a component of a remotely operated vehicle 501 .

[0084] The system 510 comprises an inspection device 515 comprising a camera 505 for recording an image of a component, identification means configured to identify the component based on the recorded image, and determination means arranged to determine a condition of the identified component using the recorded image.

[0085] In the embodiment shown, the inspection device 515 of the system 510 includes a fixed position camera 505. The system 510 shown includes a means for rotating the remotely operated vehicle 501. The means is a rotating telescopic shaft 507 adapted to engage with the frame of the vehicle 501, the shaft 507 being rotatable about its longitudinal axis. Thus, the vehicle 501 is first raised so that the vehicle wheels are disengaged from the track 108, whereupon the rotational movement of the shaft 507 turns the vehicle 501 and exposes different sections of the vehicle 501 to the fixed position camera 505. In an alternative embodiment (not shown), the camera is provided on a movable mechanical arm.

[0086] In another embodiment (not shown), the inspection device is a handheld device that includes a display.

[0087] In an alternative embodiment, the system can facilitate the component replacement process. More specifically, during the life of the storage and retrieval system 1, there may be several vehicle redesigns in addition to structural modifications of individual components resulting in many different replacement operation procedures. The system 510 will help identify the applicable procedures for a specific component / vehicle setting and present the relevant information to the operator, or in a more advanced version, guide the operator step by step through the component replacement procedure, optionally using virtual reality.

[0088] Typically, so-called high-level data for vehicle components is readily available. Examples of such high-level data are how many boxes the vehicle has lifted and / or moved, how many direction changes the vehicle has performed, or how many charging cycles the vehicle has completed. Data about the environmental conditions in which the vehicle operates, such as ambient temperature and / or pressure, is also readily available. This data is often used in machine learning-based models to improve the accuracy of component RUL predictions.

[0089] At a general level, a large amount of data needs to be collected to obtain satisfactory results. Here, the newly collected data will be used in a machine learning-based model to further improve the accuracy of this RUL prediction.

[0090] One way to implement this machine learning based model is to establish certain expected component conditions for given vehicle related intervals, such as the expected component conditions if the vehicle has been fully upgraded from 0 to 10 km, 10 km to 20 km, etc. The vehicle component images corresponding to these intervals are associated with the interval so that the machine learning based model can be trained. During this training process, each image is labeled with a corresponding high-level data value, which includes the interval with an associated RUL value. By collecting a large amount of data, the model becomes more refined, the length of the interval is reduced, and the accuracy of the RUL prediction increases.

[0091] Thus, in an inspection, initially high-level data of the vehicle and its components are collected. Then, images of the vehicle components are recorded with the aid of an image recording device. The recorded images are subsequently run through a machine learning algorithm, which will try to determine to which component-related interval the recorded images of the vehicle components fit. The estimated interval (from the model) is then compared with the actual interval (from the high-level data) to see if the values ​​disagree. The degree of difference provides information about the current quality of the model. If there is a protocol from previous component inspections, it can even be verified whether the condition of the component follows the expected trajectory or whether component wear is accelerated. Finally, the collected data (high-level data), including the labels, as well as the registered images are stored for future use.

[0092] In the foregoing description, various aspects of a method / system for determining a condition of a component of a remotely operated vehicle have been described in accordance with the present invention and with reference to illustrative embodiments. For purposes of illustration, specific numbers, systems, and configurations are set forth in order to provide a comprehensive understanding of the system and its operating principles. However, this description is not intended to be construed in a limiting sense. Various modifications and variations of the illustrative embodiments, as well as other embodiments of the method / system, that are apparent to those skilled in the art to which the disclosed subject matter belongs are deemed to be within the scope of the present invention.

[0093] Reference Numbers List

[0094] 1Storage and retrieval system

[0095] 100 frame structure

[0096] 101 Storage Unit

[0097] 102 Vertical members of the frame structure

[0098] 102' module uprights

[0099] 104 Storage Grid

[0100] 105 storage columns

[0101] 106 Storage Containers / Cargo Holders

[0102] 106' storage container specific location

[0103] 107 Stacking of storage containers

[0104] 108 Track System

[0105] 110 Parallel tracks in the first direction (X)

[0106] 111Parallel tracks in the second direction (Y)

[0107] 112 Access openings

[0108] 119 First port column

[0109] 200 modules

[0110] 201 Container handling vehicle belonging to the prior art

[0111] 201a Vehicle body of the container transport vehicle 201

[0112] 201b Drive device / wheel arrangement, first direction (X)

[0113] 201c Drive device / wheel arrangement, second direction (Y)

[0114] 301 Cantilever container handling vehicle belonging to the prior art

[0115] 301a Vehicle body of the container transport vehicle 301

[0116] 301b driving device in the first direction (X)

[0117] 301c driving device in the second direction (Y)

[0118] 304 Cantilever container handling vehicle lifting device

[0119] 304a lifting frame part

[0120] 401 Container handling vehicle belonging to the prior art

[0121] 401a Vehicle body of container transport vehicle 401

[0122] 401b driving device in the first direction (X)

[0123] 404 Cavity Remotely Operated Vehicle Lifting Device

[0124] 404a lifting frame part

[0125] 501 Remotely operated vehicle of the present invention

[0126] 505 Camera

[0127] 507 Rotating telescopic shaft

[0128] 510 System for determining the condition of a component

[0129] 515 Inspection Device

[0130] X first direction

[0131] Y second direction

[0132] Z Third direction

Claims

1. A method for determining a condition of a component of a remotely operated vehicle (501) operating on a track system (108) of an automated storage and retrieval system (1) for cargo holders (106), the method comprising: - selecting components of the remotely operated vehicle (501), - record an image of the component, - identifying the component based on the recorded image, - Determining the condition of the identified component using the recorded image.

2. The method according to claim 1, wherein: Determining the identified condition of the component includes utilizing operational data associated with the component, such as data collected during a vehicle testing phase or data collected while the vehicle is in operation.

3. A method according to any one of the preceding claims, wherein: Determining the identified condition of the component includes comparing the new recorded image of the component to a previously recorded image of the component.

4. A method according to any one of the preceding claims, wherein: Determining the condition of the identified component includes utilizing batch information associated with the component.

5. A method according to any one of the preceding claims, wherein: Determining the identified condition of the component includes taking into account environmental conditions associated with the automated storage and retrieval system (1), such as ambient temperature and / or ambient pressure.

6. A method according to any one of the preceding claims, wherein: Identifying the component based on the recorded image includes reading ID data associated with the component.

7. A method according to any one of the preceding claims, wherein: The condition of the component determined is the remaining useful life (RUL).

8. The method according to any one of claims 1 to 6, wherein: The condition of the component determined is the probability of failure of the component within a given time window.

9. A method according to any one of the preceding claims, wherein: The time for the next vehicle (501) maintenance is set based on the determined condition of the vehicle component.

10. A method according to any one of the preceding claims, wherein: In the event that the identified condition of the component is unsatisfactory, an image of a selected section of the identified component is recorded.

11. The method according to claim 10, wherein: The section of the component is selected based on at least one of: operational data associated with the component, such as data collected during a vehicle testing phase or data collected while the vehicle is in operation; batch information associated with the component; and environmental conditions associated with the automated storage and retrieval system (1), such as ambient temperature and / or ambient pressure.

12. A method according to any one of the preceding claims, wherein: An image of the component is recorded by means of a camera (505) of a handheld device associated with the operator.

13. The method according to claim 12, wherein: Information regarding the identified component is presented to the operator on a display of the handheld device.

14. The method according to any one of claims 1 to 11, wherein: An image of the component is recorded by means of at least one fixed position camera (505).

15. The method according to claim 14, wherein: The remotely operated vehicle (501) is arranged to rotate prior to recording an image of the component.

16. The method according to any one of claims 1 to 11, wherein: An image of the component is recorded by means of at least one camera (505) arranged on a movable robotic arm.

17. A method according to any one of the preceding claims, wherein: The remotely operated vehicle (501) is identified prior to selecting the component of the remotely operated vehicle (501).

18. The method according to claim 17, wherein: Identifying the remotely operated vehicle (501) includes recording an image of the vehicle (501).

19. A method according to any one of the preceding claims, wherein: The component is any one of a wheel of the vehicle (501), a gripper element, a gear box, a motor belt, a cargo holder contact sensor, and a lifting device (304, 404) for vertically transporting a cargo holder (106).

20. A system (510) for determining a condition of a component of a remotely operated vehicle (501) operating on a track system (108) of an automated storage and retrieval system (1) for cargo holders (106), the system comprising: - an inspection device (515) comprising a camera (505) for recording an image of the component, - an identification device configured to identify the component based on the recorded image, - Determination means arranged to determine the condition of said identified component using the recorded image.

21. The system (510) of claim 20, wherein: The inspection device (515) is a handheld device including a display.

22. The system (510) of claim 20, wherein: The system (510) comprises a compartment for accommodating a remotely operated vehicle (501), the compartment being provided with the inspection device (515) including the camera (505).

23. The system (510) of claim 22, wherein: The camera (505) is at least one fixed position camera.

24. The system (510) of claim 22 or 23, wherein: The remotely operated vehicle (501) is rotatable when positioned in the compartment.

25. The system (510) of claim 22, wherein: The camera (505) is at least one camera arranged on a movable mechanical arm.

26. The system (510) according to any one of claims 20 to 25, wherein: The component is any one of a wheel of the vehicle (501), a gripper element, a gear box, a motor belt, a cargo holder contact sensor, and a lifting device (304, 404) for vertically transporting a cargo holder (106).

27. An automated storage and retrieval system (1) comprising a frame structure (100) comprising a plurality of storage rows (105) for storing cargo holders (106), wherein the automated storage and retrieval system (1) comprises a system (510) for determining a status of a component of a remotely operated vehicle (501) operating on a rail system (108) by performing the steps of a method according to any one of claims 1 to 19.

Images