Component alert system

Abstract

A method to alert a user to the presence of incorrect components. Synthesised or augmented reality (AR) is used to superimpose (possibly an outline of) an identifying feature over an image of the pres

Description

Field of invention The present application relates to a method for alerting a user to the presence of an incorrect component in a container and a device for conducting such a method. In particular, the application relates to a method which implements a synthesised reality to alert a user to the presence of an incorrect component. Background Suppliers provide a multitude of different components to different manufacturers. It is important that the manufacturers are provided with the correct component parts to allow them to proceed with their manufacturing process correctly and efficiently. Typically, components are packed in a pallet and an identifier, usually a label, is applied to the pallet. This label identifies the component and allows a user to remove the pallet for delivery in accordance with the labelled pallet being requested. It is important, particularly in the automotive industry, that the correct component is received by the manufacturer and that the component is correctly labelled. For example, some components may look very similar but in fact have different features which may impact the manufacturing process and / or performance of the manufactured product (e.g., a vehicle). However, present systems may result in an incorrect label being applied to a pallet, unbeknownst to the user delivering the pallet. This can result in customers receiving incorrect components and, in some cases, result in delays in the manufacturing process. In industries which employ just-in-time manufacturing such delays can be costly. It is an aim of the present invention to address one or more of the disadvantages associated with the prior art. Summary of invention In order to mitigate at least some of the issues above, there is provided a method for alerting a user to the presence of an incorrect component in a container, the method comprising: at a device comprising a memory and one or more processors: receiving a component identifier associated with the container (in other words, receiving a component identifier of the container), wherein the component identifier corresponds to an expected component (in other words, the component identifier identified the expected component); providing a synthesised reality based on the component identifier, wherein the synthesised reality comprises an identifying feature of the expected component superimposed over a representation of a physical setting of the container; updating the synthesised reality by overlaying the identifying feature on an image of a component present in the container to provide a first overlay; in accordance with the identifying feature failing to align with the component, outputting a first alert that the component is not the expected component. The method enables an incorrect component to be identified and alerted to a user in real time. The user is typically a labeller I production worker who is responsible for adding I verifying the label to the pallet. This process ensures that the correct component is present within a container and prevents incorrect components being shipped to a customer unnecessarily thus reducing costs both financially and in terms of carbon footprint. Furthermore, some component parts are very similar in appearance such that a customer may not identify the incorrect component before manufacture. By providing a system which automatically detects incorrect components without the use of human decisionmaking, a more reliable and efficient method is provided. Optionally, wherein the synthesised reality is augmented reality. The use of augmented reality provides for an efficient and simple to use method. Optionally, wherein the component identifier is a barcode or an RFID tag. The use of a barcode as an identifier eliminates the possibility of human error whilst providing a relatively cost effective identification system. RFID tags are beneficial as they can be easily and quickly read by a scanner of sorts and store additional information. Optionally, wherein the identifying feature is displayed as an outline of a feature specific to the expected component. The identifying feature is preferably a unique feature to the identified component. The use of an outline allows a user of the device to easily align the identifying feature with the component in the container so that the device can quickly determine whether the component is the expected (correct) component. Optionally, wherein the device comprises one or more cameras and / or wherein the device is in communication with one or more cameras. The use of cameras, whether part of the device or in communication with the device, enables images to be taken throughout the process. Such images can be used in a verification process. For example, taking an image of the first overlay verifies that the component either successfully or unsuccessfully aligns with the component. In the case of successful alignment, verification is provided that the expected / correct part was indeed shipped to the customer. Furthermore, the cameras allow for logging and tracking of the components. Optionally, further comprising: in accordance with the identifying feature aligning with the component, outputting a second alert that the component is the expected component. Optionally, wherein the second alert comprises updating the synthesised reality to overlay a computer-aided design over the component to provide a second overlay. Optionally, further comprising taking an image of the second overlay. Advantageously, the second alert is provided on successful alignment and provides feedback to the user that the component is the expected / correct component. Without such a feedback mechanism, the user would be unaware whether the alignment was successful or not and thus whether the component is in the correct pallet and available for shipping. Optionally, further comprising initiating the method by receiving a user input of a unique user identifier. This provides for full traceability as to who accessed the system and completed the method. Optionally, further comprising taking an image of the first overlay. The use of cameras enables images to be taken throughout the process. Such images can be used in a verification process. For example, taking an image of the first overlay verifies that the component either successfully or unsuccessfully aligns with the component. In the case of successful alignment, verification is provided that the expected / correct part was indeed shipped to the customer. Furthermore, the cameras allow for logging and tracking of the components. Optionally, wherein receiving a component identifier associated with the container comprises scanning the component identifier associated with the container. Advantageously, scanning the component identifier provides an efficient, easy to use, and quick method of identifying the expected component in the container. There is also provided a device comprising: a memory; one or more processors; one or more displays; and one or more instructions stored in the memory which, when executed by the one or more processors, cause the device to perform any of the methods discussed above. There is also provided a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the methods discussed above. Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and / or features of any embodiment can be combined in any way and / or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and / or incorporate any feature of any other claim although not originally claimed in that manner. Brief description of the drawings Embodiments of the present invention will now be described, by way of example only, with reference to the following figures in which: Figure 1 shows a process for alerting a user to the presence of an incorrect component according to an aspect of the invention; Figure 2 shows a process for alerting a user to the presence of an incorrect or expected component according to an aspect of the invention; Figure 3 shows a schematic of a device performing an alignment method according to an aspect of the invention; Figure 4 shows a schematic of a device performing an alignment method according to an aspect of the invention; and Figures 5 shows a schematic of a device performing an alignment method according to an aspect of the invention. Figures 6a-6b show a process for alerting a user to the presence of an incorrect or expected component according to an aspect of the invention. Detailed description The present invention is a monitoring system which can be used in environments such as warehouses where multiple items are prepared and dispatched to recipients. The preparation and sending of items is typically controlled by a management system which identifies the products to be sent as well the processes which are to be followed. The present invention can integrate with, or sit separately to, existing management systems and ensures that the correct item, or items, are present before being sent. In the event that an incorrect item is identified then the process may produce an alert and furthermore may prevent the management system from completing the process for the item. A further aspect is that by recording data as part of the monitoring process a record of the process can be kept providing evidence that an item was correctly identified before dispatch. Figure 1 shows a process 100 for alerting a user to the presence of an incorrect component in a container. The process 100 is carried out at an electronic device, such as device 300 discussed in relation to Figures 3 to 5. The aim of the process is to identify whether a component located within a container is as expected. For example, to determine whether an identifier such as a label matches the physical component within the container. The process 100 begins at step 102 in which a component identifier is received at the device 300. Device 300 can be any form of known user device such as a tablet computer, laptop computer, desktop computer, smart phone, headset device etc. The component identifier is some form of indication of what component is expected to be in the container. For example, the component identifier may indicate that a particular engine part is packaged in the container. In some cases, the component identifier is in the form of a preferably unique part number, a barcode or an RFID tag, or any other form of indicia which can be read by the device 300 to identify a component or part. The component identifier can be received by the device 300 using any appropriate method known by the skilled person for entering data. For example, the component identifier may be received via user input to a keyboard (mechanical or touch screen). The user may input the part number associated with the container or the barcode number. In another example, the component identifier may be received using the device to scan the component identifier. Such scanning mechanisms are particularly useful where the component identifier is a barcode or an RFID tag. Once the component identifier has been received and the expected component identified, the process 100 proceeds to step 104. At step 104, a synthesised reality (i.e., a synthesised reality setting, a synthesised reality representation) is provided (generated) based on the component identifier. Synthesised reality includes entirely or partially computer-created content with which users can view and / or interact with. The synthesised reality can include virtual reality (VR), mixed reality (MR) or augmented reality (AR). The synthesised reality comprises the physical environment in which the device 300 is present. The physical environment in the synthesised reality may be a representation of the physical environment such as a computer-generated physical environment. The physical environment (or representation of) may show the surroundings of the device 300 including the container in a warehouse. This is described with respect to Figure 3. Shown superimposed over the representation of the physical environment is an identifying feature corresponding to the component identifier. In other words, the device 300 generates a synthesised reality including a representation of a feature specific to the component identified by the component identifier. As an example, in response to the component identifier identifying engine component A, a synthesised reality associated with engine component A is generated. The synthesised reality is presented on a display of the device 300 and at least a portion of the synthesised reality may be retrieved from a memory of the device. As an example, a portion of the synthesised reality may be retrieved from a database. The synthesised reality is preferably provided automatically once the component identifier has been received. Alternatively, the synthesised reality may be provided after a series of steps comprising user selection are completed as described in more detail with respect to Figure 6a. It is preferable for the identifying feature to be a particular feature which is preferably unique to the identified / expected component. The identifying feature is identified from the physical properties of the component. It can be, for example, a notch in the surface of the component, a protrusion from the component, or a particular arrangement of parts etc. It may also be the relative position of multiple features (e.g., surfaces, edges, features etc.) which allow the component to be identified. An aspect of the identifying feature is that because it is related to a physical property of the component, and is preferably unique to that component, it can be identified for each instance of the component. The identifying feature may be stored in a database and retrieved by the device based on the component identifier. The synthesised reality (i.e., the synthesised reality setting or representation) is then updated at step 106. The synthesised reality is updated by overlaying the representation of the identifying feature over an image of the component in the container as shown in Figure 3. For example, and as described in more detail with respect to Figure 3, an outline of the identifying feature may be overlaid on top of the image of the component in the pallet. This can be achieved by the user manoeuvring the device 300 so as to capture an image of the component in the container using a camera of the device. Alternatively, the image may be captured using one or more cameras in communication with the device 300. For example, a warehouse camera system may be positioned or controlled (either permanently or temporarily) to have a view inside the container and the image is taken by these cameras and transmitted to the device 300 for display with the updated synthesised reality. The image can be stored in a memory of the device 300 for access or retrieval at a later time. If the component in the container is not the expected component, then it will not have the identifying feature. As it does not have the identifying feature, then the identifying feature cannot be aligned with the component in the container. As the identifying feature is not present in the component in the container, then the device determines that the component in the container is not the expected component (the component identified by the component identifier). The alignment is conducted using known techniques such as snap-to-grid, snap to frame, fiducial markers, SLAM etc. The process 100 then proceeds to step 108 and an alert (e.g., notification such as visual or audio notification) is output to indicate that the component is incorrect. The warehouse management system can then prevent the user from proceeding any further with the process by automatically not progressing the process on the device and thus not allowing the container to be marked for distribution. In this way, the user is prevented from removing, or sending, the container. In this way, a user is provided with a straightforward and easy to implement method which efficiently determines and notifies that a component in the container does not match the component identifier of the container and thus there is an error. Based on this notification, a user is prevented from retrieving and delivering an incorrect component to a customer. A user can also be alerted that a component is the correct / expected component as described with respect to Figure 2. Figure 2 shows a process 200 for alerting a user to the presence of an incorrect component in a container or to the presence of an expected component in the container. Steps 202, 204, and 206 correspond to steps 102, 104, and 106 of Figure 1, respectively. Upon updating the synthesised reality at step 206 (step 106 of Figure 1), the process 200 continues to step 210 in which it is determined whether the identifying feature aligns with the component in the container. The alignment may be conducted using an outline of the identifying feature as discussed in more detail with respect to Figures 3 to 4. If the device 300 determines that the identifying feature does not align to any portion of the component in the container, the process 200 proceeds to step 208 and an alert is output to notify the user of the unexpected component. Step 208 corresponds to step 108 of Figure 1. If the device 300 determines that the identifying feature in the synthesised reality user interface does align with at least a portion of the component in the container, a second alert is output to the user at step 212. This second alert may be in the form of a visual or audible notification. The second alert notifies the user that the component in the container matches the component identifier and the component has been packaged in the right container. The user is then able to confidently retrieve the container and continue with the process of delivering the component to a customer. In this way, the process 200 works a as a verification method for the components. Figure 3 illustrates the alignment process carried out at a device 300. The device 300 can be any appropriate user device such as a tablet computer, laptop computer, desktop computer, smart phone etc. The device 300 is handled by a user when completing any of processes 100, 200 or 600. The device 300 displays (via a display) a synthesised reality. This can be in the form of a synthesised reality setting or a synthesised reality representation that is displayed on device 300. The synthesised reality user interface comprises a representation of the physical environment 306 in which the device 300 is located. For example, the physical environment 306 may comprise the container and other objects within a warehouse housing the container. The synthesised reality also comprises an image of a component 304. The image of the component 304 may include the entirety of the component 304 or may include only a portion of the component 304. Preferably, the image comprises the entirety of the component 304 to ensure all features of the component 304 are available for identification. The component 304 comprises an identifying feature 308, for example, the elongated portion with the protrusions. The identifying feature 308 is a feature of the component 304 which is particular to that component 304 such that the component 304 can be identified from the identifying feature 308. Preferably, the identifying feature 308 is a unique feature. The synthesised reality user interface also displays an identifying feature 310 of the expected component. For example, if the component identifier identified piston X as the expected component in the container, then the identifying feature 310 would be the identifying feature of piston X. The synthesised reality is updated to overlay the identifying feature 310 over the image of the component 304. As shown in Figure 3, the identifying feature 310 of the expected component cannot be successfully overlayed on the image of the component 304 as the component 304 in the container does not comprise the identifying feature 310. As such, the device 300 determines that the expected component with identifying feature 310 is not located within the container and that the component 304 in the container is unexpected / incorrect. Figure 4 illustrates an alignment process carried out at device 300 where the alignment is successful. In this case, the identifying feature 410 corresponding to the component identifier matches the identifying feature 308 of component 304. This is shown in the synthesised reality by the identifying feature 410 successfully aligning with the identifying feature 308 of the image of the component 304 in the container. Following the successful alignment of Figure 4, a further optional step can be carried out as shown by Figure 5. Once the alignment has been successful, the device 300 provides user feedback to the user to indicate that the component identifier matches the component in the container. This can be provided in the form shown in Figure 5 in which a computer-aided design (CAD) 512 of the component 304 is snapped to the image of the component 304. The CAD can be the entirety of the component 304 or only a portion of the component 304, such as the identifying feature 308. By providing such visual feedback, the user is alerted to the fact that the component identifier corresponds to the component in the container such that the component has been packaged and labelled correctly. Without such a mechanism, the user would not be aware whether the alignment was successful. Furthermore, this step acts as a further verification that the component identifier matches the component in the container. In the example of Figure 5, if the CAD 512 of the component 304 did not align with or overlay correctly to the image of the component 304, the user is alerted that the identifier does not match the component. In this way, the visual feedback can act as a second verification step to provide further robustness that the identifier is indeed correct. This provides a greater level of confidence that the component identifier is correct for the component in the container. This may be particularly useful where the component identifier is not unique to a single component. For example, there may be a case where the identifying feature applies to two components. In one example, identifying feature X corresponds to component Y and to component Z. In this example, the synthesised reality generated is associated with component Y but the component in the container is component Z. However, as both components have the same identifying feature X, the first verification step is successful despite the component being incorrect. Where the further verification step is implemented, the user may be alerted that although the first verification step was successful, the second verification step is not. As such, the second verification step provides increased confidence that the component identifier complies with the component in the container and means that the component identifier does not need to be unique. Figures 6a and 6b show a process 600 for alerting the user to an expected or unexpected component in the container. Process 600 comprises the steps from processes 100 and 200 with some additional optional steps (optional steps 602, 606, 608, 610, 622, 624, and 626). The alert process 600 begins with step 602 in which a user identifier is received at the device 300. The user identifier is a unique identification mechanism for each user of the device. For example, each employee may have a personal username and password which allows them to login to the system. The unique identification may be in the form of a username, password, login number, RFID tag, biometric etc. If the user identifier is authorised, the process 600 proceeds to step 604. Step 604 corresponds to steps 102 and 202 discussed with respect to Figures 1 and 2. This is the step of receiving a component identifier. Once the device 300 has received the component identifier, the device 300 displays a component program corresponding to the component identifier. For example, if the component identifier corresponds to a valve Y, the component program loaded by device 300 is the one specific to valve Y. The component program may be stored in memory and retrieved by the device 300 in response to the component identifier. As an example, the component program may be retrieved from a database comprising a plurality of component programs. The component program is displayed and is configured to be selected by a user at step 608. After selecting the component program at step 608, a further user input is received at step 610 to start the inspection of the component in the container. The user inputs at steps 602, 608, and 610 can be received in any known manner. For example, via a keyboard, a touch screen, a computer mouse, a trackpad etc. In response to the device 300 receiving user input to start the inspection, the process 600 proceeds to step 612. Step 612 corresponds to steps 104 and 204 discussed with respect to Figures 1 and 2. This is the step of providing a synthesised reality. Step 614 corresponds to steps 106 and 206 discussed with respect to Figures 1 and 2. This is the step of updating the synthesised reality. Step 616 corresponds to step 210 discussed with respect to Figure 2. This is the step of determining whether the identifying feature aligns with an image of the component in the container. Step 618 corresponds to steps 108 and 208 discussed with respect to Figures 1 and 2. This is the step of outputting a first alert upon determining that the identifying feature does not align with the image of the component in the container. Step 620 corresponds to step 212 discussed with respect to Figure 2. This is the step of outputting a second alert upon determining that the identifying feature does align with the image of the component in the container. If the alignment is successful and the second alert is output at step 620, some further steps can be completed to provide further assurance and trackability. At step 622, a picture is taken of the label on the container. The image can be taken using the device 300 or an alternative camera in communication with the device 300. The image can be stored in a memory of the device 300 so that it can be accessed and retrieved at a later time. This image can be used to confirm the label and the container that the process was carried out for. At step 624, a ticket identifier is received by the device 300. The ticket identifier is a form of identification used to track and identify a process 100, 200, 600 carried out on the container. The ticket identifier may be in the form of a barcode, a ticket number etc. The ticket identifier can be received using any of the mechanisms discussed in relation to receiving the component identifier. For example, the ticket identifier may be scanned by the device 300. Once the ticket identifier has been received at step 624, the user can complete the method at step 626 by providing a further input to finish the process 100, 200, 600. The above embodiments are provided as examples only. The scope of the invention is defined by the appended independent claims. The invention covers all modifications, variations and equivalents as fall within the scope of the appended independent claims.

Claims

1. A method for alerting a user to the presence of an incorrect component in a container, the method comprising:at a device comprising a memory and one or more processors:receiving a component identifier associated with the container, wherein the component identifier corresponds to an expected component;providing a synthesised reality based on the component identifier, wherein the synthesised reality comprises an identifying feature of the expected component superimposed over a representation of a physical setting of the container;updating the synthesised reality by overlaying the identifying feature on an image of a component present in the container to provide a first overlay; andin accordance with the identifying feature failing to align with the component, outputting a first alert that the component is not the expected component.

2. The method of claim 1, wherein the synthesised reality is augmented reality.

3. The method of any preceding claim, wherein the component identifier is a barcode or an RFID tag.

4. The method of any preceding claim, wherein the identifying feature is displayed as an outline of a feature specific to the expected component.

5. The method of any preceding claim, wherein the device comprises one or more cameras and / or wherein the device is in communication with one or more cameras.

6. The method of any preceding claim, further comprising:in accordance with the identifying feature aligning with the component, outputting a second alert that the component is the expected component.

7. The method of claim 6, wherein the second alert comprises updating the synthesised reality to overlay a computer-aided design over the component to provide a second overlay.

8. The method of claim 7 when dependent on claim 5, further comprising taking an image of the second overlay.

9. The method of any preceding claim, further comprising initiating the method by receiving a user input of a unique user identifier.

10. The method of any preceding claim when dependent on claim 5, further comprising taking an image of the first overlay.11.The method of any preceding claim, wherein receiving the component identifier associated with the container comprises scanning the component identifier associated with the container.

12. A device comprising:a memory;one or more processors;one or more displays; andone or more instructions stored in the memory which, when executed by the one or more processors, cause the device to perform any of the methods of claims 1-11.

13. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the methods of claims 1-11.

Images