Optimizing the verification of automated auditing in equipment installation

Abstract

A system to verify the correctness of equipment installation configured to perform the steps of: accessing a digital model of the equipment; receiving a representation of the equipment indicating a configuration of the equipment, Accessing installation-specific metadata associated with the installation of the equipment; Predicting at least one behaviour of the equipment based on metadata associated with installation of the equipment; Accessing service information comprising at least one measured behaviour of the equipment; Comparing the representation with the model to determine a first degree of conformity of the equipment with the model of the equipment; if the first degree of conformity is within a first threshold degree of conformity, comparing the at least one predicted behaviour with the at least one measured behaviour to determine a second degree of conformity of the equipment with the service information; and if the second degree of conformity is within a threshold second degree of conformity, verifying to an operator that the equipment is correctly installed.

Description

[0001] Optimizing the Verification of Automated Auditing in Equipment Installation

[0002] The present invention relates to the installation of equipment and the automated verification of the efficacy of such installation.

[0003] The installation of physical equipment by installation operatives such as engineers, service personnel and installers are susceptible to inconsistencies of approach, procedure and / or inconsistent application of quality assurance measures or checks. Consequently, equipment can be installed with varying degrees of correctness ranging from entirely correct to entirely incorrect with a continuous spectrum of measures of correctness therebetween, any of which may be objectively defined or measured according to a scale, rating or other scheme for measuring or verifying correctness of installation. In particular, while installation procedures may have been substantially followed correctly by an operative, particular characteristics of the installation process, the installed equipment itself as handled or treated by an operative, other equipment or resources associated with the equipment or an environment of installation of the equipment may render the installed equipment susceptible to a subsequently curtailed lifespan, a curtailed or reduced efficacy or performance, a susceptibility to progressive or sudden degradation in utility, suitability or performance, or other such detriments. Notably, such detriments may not be readily identified as the equipment can be seen to be immediately operable on installation, especially where such detriments relate to a failure to achieve a potential performance or a progressive decline in performance of efficacy over time.

[0004] Such equipment installation is common in utilities industries, such as industries that have transmission and distribution networks for the transport, provision, communication or conveyance of a utility such as power (including electricity), gas (including natural gas), liquid (including water), sewage and communications facilities (including fixed-line, wireless and / or mobile telephony and network connections such as broadband services). Such networks are comprised of network infrastructure including means and mechanisms for the transmission of the utility. Such infrastructure can include equipment that is deployed as part of the provision of the utility, including equipment deployed at utility provider premises, business or consumer premises, and at locations therebetween.

[0005] For example, the telecommunications industry provides network communications services to businesses and consumers including services such as fibre optic network connections, the provision of which involves the installation of equipment. Incorrect installation of such equipment can lead to the potential early failure, shortened life, reduced performance or other detriments. Therefore, operatives audit an installation to ascertain the degree of correctness of the equipment installation. In telecommunications installations, a considerable proportion of all equipment installation audits reveal a critical error in the installation. A critical error means that there is a need for revisit or repair by the installation operative. For telecommunication installations, such as installation of a customer splice point at which fibre optic cables can be spliced for provision of fibre optic services to a customer’s premises, the number of installations outnumbers the number of auditor operatives so critical errors will not be addressed.

[0006] Accordingly, there is a challenge to address inconsistent installation of equipment to ensure efficacious and suitably performing equipment on installation. There is also a need to provide a mechanism for improved auditing of the installed equipment.

[0007] The present invention accordingly provides, in a first aspect, a computer-implemented method to verify the correctness of equipment installation, the method comprising: accessing a digital model of the equipment; receiving a representation of the equipment indicating a configuration of the equipment, Accessing installation-specific metadata associated with the installation of the equipment; Predicting at least one behaviour of the equipment based on metadata associated with installation of the equipment; Accessing service information comprising at least one measured behaviour of the equipment; Comparing the representation with the model to determine a first degree of conformity of the equipment with the model of the equipment; if the first degree of conformity is within a first threshold degree of conformity, comparing the at least one predicted behaviour with the at least one measured behaviour to determine a second degree of conformity of the equipment with the service information; and if the second degree of conformity is within a threshold second degree of conformity, verifying to an operator that the equipment is correctly installed.

[0008] In one arrangement of the method, the representation is generated by at least one detector.

[0009] In one arrangement of the method, the representation is generated by at least one detector.

[0010] In one arrangement of the method, the method further comprising the step of, if the second degree of conformity is within a threshold second degree of conformity, outputting the representation and associated metadata to a data store as training data accessible for training the digital model.

[0011] In one arrangement of the method, the metadata comprises one or more of: temporal metadata, engineer metadata and location metadata. The present invention accordingly provides, in a second aspect, a data processing system comprising a processor configured to carry to perform the steps of: accessing a digital model of the equipment; receiving a representation of the equipment indicating a configuration of the equipment, Accessing installation-specific metadata associated with the installation of the equipment; Predicting at least one behaviour of the equipment based on metadata associated with installation of the equipment; Accessing service information comprising at least one measured behaviour of the equipment; Comparing the representation with the model to determine a first degree of conformity of the equipment with the model of the equipment; if the first degree of conformity is within a first threshold degree of conformity, comparing the at least one predicted behaviour with the at least one measured behaviour to determine a second degree of conformity of the equipment with the service information; and if the second degree of conformity is within a threshold second degree of conformity, verifying to an operator that the equipment is correctly installed.

[0012] The present invention accordingly provides, in a third aspect, a computer program element comprising computer program code to, when loaded into a computer system and executed thereon, cause the computer to perform the steps of a method described above.

[0013] Exemplary arrangements of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0014] Figure 1 is a block diagram a computer system suitable for the operation of embodiments of the present invention;

[0015] Figure 2 is a flowchart of an exemplary method in accordance with an exemplary arrangement of the present invention;

[0016] Figure 3 is a component diagram of an automated equipment installation verification system to automatically verify correctness of installation actions of an operative installing an item of equipment in accordance with an exemplary arrangement of the present invention;

[0017] Figure 4 is a component diagram of an automated equipment installation verification system to automatically verify correctness of installation actions of an operative installing an item of equipment in accordance with an exemplary arrangement of the present invention;

[0018] Figure 1 is a block diagram of a computer system suitable for the operation of embodiments of the present invention. In particular logic units as herein described may be implemented as physical or virtual generalised computer systems or dedicated or bespoke processing systems, such as systems configured to execute program code whether software or hardware-encoded such as via stored instructions, firmware or a combination thereof. A central processor unit (CPU) 102 is communicatively connected to a storage 104 and an input / output (I / O) interface 106 via a data bus 108. The storage 104 can be any read / write storage device such as a random-access memory (RAM) or a non-volatile storage device. An example of a non-volatile storage device includes a disk or tape storage device. The I / O interface 106 is an interface to devices for the input or output of data, or for both input and output of data. Examples of I / O devices connectable to I / O interface 106 include a keyboard, a mouse, a display (such as a monitor) and a network connection.

[0019] If an automatic audit can be carried out in few minutes while the field engineer is still onsite then it is possible to advice the field engineer on how to rectify detects and avoid the cost of a revisit to the installation site.

[0020] The present invention provides an optimised method to analyse a representation of the equipment by making use of installation metadata (such as time / duration, engineer and / or location, etc.), e assume that a human / robot carries out an equipment installation and interacts with a system that aids through an Al-based learning system, e.g. an engineer or robot performing a task with remote assistant.

[0021] The following is an exemplary approach of the present invention that can be used to optimise an automated auditing of equipment installations:

[0022] Stage 1 : The model, e.g. an Al-based system, obtains media data (e.g., 2D images or 3D models) interpreted as a representation of a scene provided by human / robot operatives that may also select a relevant business application, for example a scene of a fibre optic corresponds to the relevant business application of telecommunications installations. The input media data of the scene (e.g., installed equipment / devices) is transformed into elementary objects such as video frames, images.

[0023] Stage 2: The representation and its regions of interest are inferred in a digital model of the equipment as follows:

[0024] The present invention makes use of reference model of the digital object, i.e., the digital model, with known measurements of length, height and / or location of features of interest (e.g. in mm). The reference model holds features’ positions and relative placement / distance of such features in the installed equipment. This digital model also prescribes how distance between features are interpreted, for example in one model the distance between two screws may be interpreted as distance between the centre points of their boundaries, while in another the distance may be between their nearest boundaries.

[0025] Step 2.1. This step applies detectors to detect the installed equipment within the elementary object (e.g. image), with models trained to identify a specific part of the elementary object (e.g. corner, edge of device, border) as follows: we assume that there exist xy axes with metric units which are interpreted as the edges of the representation (e.g. edge of the whole image); a) Apply a set of modules to pre-process the representation (e.g. image deblurring, enlarging); b) Apply an iterative process that at each iteration i, employs edge-finding detectors (e.g. machine learning or pixel-based detectors) to the elementary object to identify the position of three corners, where a corner is defined as the location where two edges meet. This can be extended / generalised by seeking to identify W > 0 unique features which always exists and from which all others can be derived via the digital model c) Assume that detectors find top-right corner, bottom-right corner, and top-left corner in the xy axes. Applying the approximation method described earlier, it is inferred that the expected location in the xy axes system of all the remaining features’ regions based on the known digital model, e.g. infer that a screw should be in the xy axes at a particular region; d) Apply each feature (associated with the installed equipment) detector to the representation in the region where such feature is expected following step in c) (e.g. apply a screw detector in the region where it is expected). If the feature is found, use this detector’s output to identify its location in the xy axes e) Calculate the deviation of the location of each feature obtained in c) and d) in the xy axes. In case the feature is not found this deviation will be large; f) Calculate the total standard of all features’ deviations in e) for iteration I and given pre-processors modules applied at that iteration; g) An optimisation process or heuristic models or machine learning process operates over all modules in a) and iterations and selects the optimal digital model and regions (i.e. defined by coordinates in xy axes), as inferred in steps b), c) d), which minimises the standard deviation. This can be used for further processing. Stage 3: Extract regions of the optimised digital model that are expected to contain features and use the business logic to aggregate these features if appropriate.

[0026] Stage 4: Pre-trained detector(s) (e.g. machine learning or pixel-based detectors) are applied to search for features in such regions, aggregated features where appropriate. These aggregated detectors’ output and business logic can give an indication of the accuracy of an installation, e.g. with combined individual features’ detection confidence if appropriate.

[0027] Stage 5: Installation metadata (such as time / duration, engineer and / or location) is used to further verify the accuracy of the installation. The duration of an installation is an important element to include in the metadata and in the verification process to compare actual duration compared to expected duration - this could be applied for step-wise and / or overall installation time / duration.

[0028] Stage 6: The output of Stages 4 and 5 returned to the human / robot as the outcome of the automated auditing. The outcome of Stage 4 in particular will provide localised feedback of audit enabling the human / robot real-time rectification.

[0029] The benefits of the approach in this invention are that multiple detectors and accompanying parallel classifiers allow for a higher level of interpretability and targeted detection performance improvements in an environment of changing business logic, reinforced by in-life measurable outcomes. In case of features’ detection with low confidence, the present invention can provide newly classified labelled data which are derived by in-life measurable outcomes (e.g. poorquality measures) which improve detection accuracy.

[0030] Figure 2 is a flow diagram of an exemplary method of training a model for detecting correctly installed equipment.

[0031] Starting as step 210, a digital model of the equipment is accessed. The digital model of the equipment can be image data, video data or audio data associated with the installed equipment. For example, the digital model of the equipment is image data, and in one example, the installed equipment is a fibre optic cable with a bend in it. The digital model can be accessed or generated from a data store.

[0032] In step 220, the representation of the equipment indicating a configuration and / or status of the equipment is received. The representation can be in the same format as the digital model, e.g., if the digital model is a 3D image scan, the representation is also a 3D image scan. The representation may be captured in situ by the installation operative. The representation of the equipment at least indicates a configuration of the equipment such as by an optical, acoustic, thermal or other representation generated by a corresponding sensor. For example, the representation may be generated based on a sensor such as a camera, a 3D scanner, a LiDAR sensor, a sound sensor or detector, a thermal sensor, a pressure gauge or other sensor capable of generating such a representation indicating a configuration of the equipment.

[0033] In step 230, installation-specific metadata associated with the installation of the equipment is accessed. The installation-specific metadata may include at least one of the temporal metadata (e.g. duration of installation process), engineer metadata (e.g. name / experience of installation operative) and location metadata (e.g., GPS location).

[0034] In step 240, at least one behaviour of the equipment based on metadata associated with installation of the equipment is predicted. For example, temporal metadata that implies that the installation of the equipment was completed significantly more quickly than the expected duration to correctly install the equipment will be assigned a predicted behaviour of likely failure. The predicted behaviours and associated implications may be stored in a data store..

[0035] In step 250, service information comprising at least one measured behaviour of the equipment is accessed. For example, the installation operative or auditing operative may measure that the bent fibre results in a 25% reduced rate receiving of packets sent over the cable.

[0036] In step 260, a comparison is made with the representation with the model to determine a first degree of conformity of the equipment with the model of the equipment.

[0037] In step 270, if the first degree of conformity is within a first threshold degree of conformity, comparing the at least one predicted behaviour with the at least one measured behaviour to determine a second degree of conformity of the equipment with the service information.

[0038] In step 280, if the second degree of conformity is within a threshold second degree of conformity, it is verified to an operator that the equipment is correctly installed.

[0039] Figure 3 is a component diagram of an automated equipment installation verification system to automatically verify correctness of installation actions of an operative installing an item of equipment in accordance with an exemplary arrangement of the present invention. The system is depicted as a client / server system having a server device 300 and a client device 302. It will be appreciated that such a separation of the system into multiple devices is purely exemplary and a singular or multiplicity of devices may be used. The client device 302 includes or is constituted by a logic unit 310 such as a general purpose or dedicated computing or processing device. The client device 302 further includes, or has associated, at least one sensor 308 for sensing physical acts performed by an operative installing equipment. The sensor 308 thus generates data corresponding to each act a sequence of acts performed by the operative, such as by way of a sensor as described above. The logic unit 310 of the client device executes at least one classifier trained to determine a degree of correctness of an act of the operative based on the data corresponding to the act provided by the sensor 308. In one arrangement, the logic unit 310 is further operable communicate an output of the classifier to a rule engine 306 of a server device 300.

[0040] The server device 300 includes or is constituted by a logic unit 304 such as a general purpose or dedicated computing or processing device that executes a rule engine 306. The rule engine 306 can be provided as a series of program code or firmware, or can be provided as a dedicated hardware rule engine for the provision and execution of rules. The rule engine 306 applies at least one rule to an output of the at least one classifier for each of a sequence actions of the operative in the installation of the equipment, as previously described, to generate an indication of a degree of correctness of the actions performed by the operative.

[0041] While the logic unit 304 of the server device and the logic unit 310 of the client device are depicted as separate, it will be appreciated by those skilled in the art that such separation is optional and the logic units can be combined into a composite logic unit. Further, the client 302 and server 300 devices can be communicatively connected such as via a communications interface such as a wired, wireless, bus or other suitable interface.

[0042] Figure 4 is a component diagram of an automated equipment installation verification system to automatically verify correctness of installation actions of an operative installing an item of equipment in accordance with an exemplary arrangement of the present invention. The arrangement of Figure 4 shares many features in common with the arrangement of Figure 3 and these will not be repeated here. The exemplary arrangement of Figure 4 shows a client 402 that may be provided to an installation operative performing an installation of equipment at a remote location of the installation, and a server 400 provided at a different location such as a data centre or office or the like. The client 402 and server400 include communications interfaces 426 and 420 respectively to provide a communicative connection therebetween such as via a cellular data network and / or wired or wireless network. Via the arrangement of Figure 4, the client further includes a visual display 422 such as a screen, indicator or other visual means, for providing the operative with information arising from the automated equipment installation verification system, such as indications of conformity of one or more installation actions and / or particular installation actions having degree of conformity that does not meet a threshold degree of correctness. A sensor 408 is depicted as part of the client device 402 though it will be appreciated that the sensor 408, such as a camera, LiDAR sensor, sound sensor or the like, may be provided as a separate device or component of a device such that data generated by the sensor 408 corresponding to an act of the operative is communicable to the client device 402 for processing by its logic unit 410, or communicable to the server device 400 for processing by its logic unit 404. In one example, the client device 402 is a portable computing device such as a smartphone, tablet, laptop computer or the like. Thus, in use, the sensor 408 generates data characterising each of a set of acts of the operative in the installation of the equipment. The client device executes the classifier 424 to classify the data to determine a degree of correctness of an act of the operative in the sequence of stepwise actions. The classifier output for each of a sequence of acts of the operative is processed by a rule engine 406 of the server device 400 to apply at least one rule to generate an indication of a degree of correctness of the sequence of acts. The degree of correctness of the sequence of acts of the operative, and in some arrangements, one or more individual acts of the operative, is communicated to the client device 402 in order that it may be displayed on the visual display 422 to inform the operative.

[0043] Insofar as embodiments of the invention described are implementable, at least in part, using a software-controlled programmable processing device, such as a microprocessor, digital signal processor or other processing device, data processing apparatus or system, it will be appreciated that a computer program for configuring a programmable device, apparatus or system to implement the foregoing described methods is envisaged as an aspect of the present invention. The computer program may be embodied as source code or undergo compilation for implementation on a processing device, apparatus or system or may be embodied as object code, for example.

[0044] Suitably, such a computer program is stored on a carrier medium in machine or device readable form, for example in solid-state memory, magnetic memory such as disk or tape, optically or magneto-optically readable memory such as compact disk or digital versatile disk etc., and the processing device utilises the program or a part thereof to configure it for operation. The computer program may be supplied from a remote source embodied in a communications medium such as an electronic signal, radio frequency carrier wave or optical carrier wave. Such carrier media are also envisaged as aspects of the present invention.

[0045] It will be understood by those skilled in the art that, although the present invention has been described in relation to the above described example embodiments, the invention is not limited thereto and that there are many possible variations and modifications which fall within the scope of the invention.

[0046] The scope of the present invention includes any novel features or combination of features disclosed herein. The applicant hereby gives notice that new claims may be formulated to such features or combination of features during prosecution of this application or of any such further applications derived therefrom. In particular with reference to the appended claims, features from dependent claims may be combined with those of the independent claims and features from respective independent claims may be combined in any appropriate manner and not merely in the specific combinations enumerated in the claims.

Claims

CLAIMS1 . A computer-implemented method to verify the correctness of equipment installation, the method comprising:- Accessing a digital model of the equipment;- Receiving a representation of the equipment indicating a configuration of the equipment,- Accessing installation-specific metadata associated with the installation of the equipment;- Predicting at least one behaviour of the equipment based on metadata associated with installation of the equipment;- Accessing service information comprising at least one measured behaviour of the equipment;- Comparing the representation with the model to determine a first degree of conformity of the equipment with the model of the equipment; if the first degree of conformity is within a first threshold degree of conformity, comparing the at least one predicted behaviour with the at least one measured behaviour to determine a second degree of conformity of the equipment with the service information; and if the second degree of conformity is within a threshold second degree of conformity, verifying to an operator that the equipment is correctly installed.

2. The method of claim 1 , wherein the representation is generated by at least one detector.

3. The method of claim 1 or 2, further comprising the step of, if the second degree of conformity is within a threshold second degree of conformity, outputting the representation and associated metadata to a data store as training data accessible for training the digital model.

4. The method of any preceding claim, wherein the metadata comprises one or more of: temporal metadata, engineer metadata and location metadata.

5. A computer program element comprising computer program code to, when loaded into a computer system and executed thereon, cause the computer to perform the steps of a method as claimed in any of claims 1 to 4.

6. A data processing system comprising a processor configured to perform the steps of:- Accessing a digital model of the equipment;Receiving a representation of the equipment indicating a configuration of the equipment,- Accessing installation-specific metadata associated with the installation of the equipment;- Predicting at least one behaviour of the equipment based on metadata associated with installation of the equipment;- Accessing service information comprising at least one measured behaviour of the equipment;- Comparing the representation with the model to determine a first degree of conformity of the equipment with the model of the equipment; if the first degree of conformity is within a first threshold degree of conformity, comparing the at least one predicted behaviour with the at least one measured behaviour to determine a second degree of conformity of the equipment with the service information; and if the second degree of conformity is within a threshold second degree of conformity, verifying to an operator that the equipment is correctly installed.

7. The system of claim 6, wherein the representation is generated by at least one detector.

8. The system of claim 6 or 7, further comprising the step of, if the second degree of conformity is within a threshold second degree of conformity, outputting the representation and associated metadata to a data store as training data accessible for training the digital model.

9. The system of any of claims 6 to 8, wherein the metadata comprises one or more of: temporal metadata, engineer metadata and location metadata.

Images