Utility infrastructure mapping

Abstract

There is provided a method and apparatus for utility infrastructure mapping, comprising acquiring (S101) ground-level images of a first geographic area, performing object detection (S102) to identify first utility infrastructure objects, and determining (S104) a first grid comprising indications of the first utility infrastructure objects at locations. Top- down images of a second geographic area are acquired (S105), object detection is performed (S106) on the top-down images to identify second utility infrastructure objects, and a second grid is determined (S108) comprising indications of the second utility infrastructure objects. Sensor fusion is performed (S109) to determine an integrated grid by integrating the first and second grids. An above-ground utility infrastructure map is determined (S110) based on the integrated grid. Generative modelling is performed (S111) to determine an underground utility infrastructure map based on the above- ground utility infrastructure map.

Description

[0001] Utility Infrastructure Mapping

[0002] Field

[0003] The present disclosure relates to utility infrastructure mapping, and particularly a method and apparatus for utility infrastructure mapping.

[0004] Background

[0005] The location of existing underground and overground utility infrastructure (e.g. electricity power lines, electrical substations, gas and water pipes and other infrastructure related landmarks) is often poorly known. This creates significant risk, for example in infrastructure and highway construction projects. 3D models of underground utility infrastructure can be created with remote sensing technology such as ground penetrating radar, but this approach is time-consuming and expensive.

[0006] Furthermore, the growing complexity and expansion of utility networks have made it increasingly difficult to maintain accurate records and maps of infrastructure assets. Outdated or inaccurate information can lead to inefficient maintenance and repairs, potentially causing service disruptions and safety hazards. The problem is compounded by the lack of timely and comprehensive data for decision-makers in utility management.

[0007] An objective is to overcome the aforementioned challenges, amongst others.

[0008] Summary

[0009] In a first aspect, there is provided a computer-implemented method of utility infrastructure mapping, the method comprising: acquiring one or more ground-level images of a first geographic area; performing object detection on the ground-level images to identify one or more first utility infrastructure objects; determining a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; acquiring one or more top-down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; performing object detection on the top-down images to identify one or more second utility infrastructure objects; determining a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; performing sensor fusion to determine an integrated grid by integrating the first grid and the second grid; determining an above-ground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and performing generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the aboveground utility infrastructure map.

[0010] Optionally, the ground-level images are captured using one or more low-level cameras, and / or wherein the top-down images are aerial images and / or satellite images.

[0011] Optionally, the first utility infrastructure objects comprise objects that pertain to utility infrastructure that is visible at ground level, and / or wherein the second utility infrastructure objects comprise objects that pertain to utility infrastructure that is visible from an aerial and / or satellite view.

[0012] Optionally, the first utility infrastructure objects comprise one or more of electrical power lines, electrical substations, manholes covers, water infrastructure, and sewer infrastructure; and / or wherein the second utility infrastructure objects comprise power lines and / or electrical substations.

[0013] Optionally, the performing object detection on the ground-level images further comprises performing image segmentation to localise and classify one or more first utility infrastructure objects; and / or wherein performing object detection on the top-down images further comprises performing image segmentation to localise and classify one or more second utility infrastructure objects.

[0014] Optionally, the indications of the one or more first utility infrastructure objects at locations on the first grid represent a probability score of such a utility infrastructure object being present at the location; and / or wherein the indications of the one or more second utility infrastructure objects at locations on the second grid represent a probability score of such a utility infrastructure object being present at the location.

[0015] Optionally, the method further comprises: applying georeferencing to the ground-level images, wherein the determined locations of the indications of the one or more first utility infrastructure objects on the first grid are determined based on the georeferencing applied to the ground-level images; and applying georeferencing to the top-down images, wherein the determined locations of the indications of the one or more second utility infrastructure objects on the second grid are determined based on the georeferencing applied to the top-down images.

[0016] Optionally, applying georeferencing to the ground-level images comprises mapping the ground-level images to geographical coordinates; and / or wherein applying georeferencing to the top-down images comprises identifying a plurality of ground control points with established locations on the top-down image, and deploying a curve fit to produce a parametric formula referencing other points in the top- down image.

[0017] Optionally, the method further comprises: pre-processing the ground-level images to remove noise, correct distortions and / or standardize formats; and / or pre-processing the top-down images to remove noise, correct distortions and / or standardize formats. Optionally, the top-down images comprise thermal and / or multispectral satellite imagery, and the method further comprises detecting anomalies in electromagnetic fields, and / or utilising visual information hidden in red-green-blue images, based upon the thermal and / or multispectral satellite imagery.

[0018] Optionally, the method further comprises applying Normalized Difference Vegetation Index analysis to the top-down images to determine areas with potential electromagnetic field disturbances based on changes in vegetation health identified from the Normalized Difference Vegetation Index analysis.

[0019] Optionally, the top-down images comprise thermal imagery, and the method further comprises detecting temperature anomalies associated with electromagnetic fields.

[0020] Optionally, the method further comprises combining the integrated grid with data from electromagnetic field sensors that measure electromagnetic fields in vicinities around power lines and / or electrical substations, and correlating the data from the electromagnetic field sensors with data from the ground-level images and / or top-down images to identify areas with electromagnetic field disturbances or deviations from expected levels.

[0021] Optionally, the method further comprises outputting an alert when, based on the aboveground utility infrastructure map and underground utility infrastructure map, and the determined locations of utility infrastructure objects, there is a risk of damaging existing underground utility infrastructure.

[0022] In a second aspect, there is provided a utility infrastructure mapping apparatus comprising: a ground-level image acquisition module configured to acquire one or more ground-level images of a first geographic area; an object detection module configured to perform object detection on the groundlevel images to identify one or more first utility infrastructure objects; a grid determination module configured to determine a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; a top-down image acquisition module configured to acquire one or more top- down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; an object detection module configured to perform object detection on the top- down images to identify one or more second utility infrastructure objects; a grid determination module configured to determine a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; a sensor fusion module configured to perform sensor fusion to determine an integrated grid by integrating the first grid and the second grid; an above-ground map determination module configured to determine an aboveground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and a generative modelling module configured to perform generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the above-ground utility infrastructure map.

[0023] Optionally, the utility infrastructure mapping apparatus of the second aspect can include the optional features of the first aspect, as appropriate.

[0024] In a third aspect, there is provided a computer-readable medium storing instructions thereon, that when executed by one or more processors, cause the one or more processors to perform the following steps: acquiring one or more ground-level images of a first geographic area; performing object detection on the ground-level images to identify one or more first utility infrastructure objects; determining a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; acquiring one or more top-down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; performing object detection on the top-down images to identify one or more second utility infrastructure objects; determining a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; performing sensor fusion to determine an integrated grid by integrating the first grid and the second grid; determining an above-ground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and performing generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the aboveground utility infrastructure map.

[0025] Optionally, the computer-readable medium of the third aspect can include the optional features of the first aspect, as appropriate.

[0026] In a fourth aspect, there is provided a means for utility infrastructure mapping, the means for utility infrastructure mapping comprising: means for acquiring one or more ground-level images of a first geographic area; means for performing object detection on the ground-level images to identify one or more first utility infrastructure objects; means for determining a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; means for acquiring one or more top-down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; means for performing object detection on the top-down images to identify one or more second utility infrastructure objects; means for determining a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; means for performing sensor fusion to determine an integrated grid by integrating the first grid and the second grid; means for determining an above-ground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and means for performing generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the aboveground utility infrastructure map.

[0027] Optionally, the means for utility infrastructure mapping of the fourth aspect can include the optional features of the first aspect, as appropriate.

[0028] Brief Description of Drawings

[0029] Examples of the disclosure are now described, with reference to the drawings, in which:

[0030] FIG. 1 is a flow diagram of a computer-implemented method of utility infrastructure mapping;

[0031] FIG. 2 is a block diagram of a utility infrastructure mapping apparatus; and

[0032] FIG. 3 is a high-level block diagram of an apparatus suitable for implementing various aspects of the disclosure.

[0033] Detailed Description

[0034] In the present disclosure, utility infrastructure can be considered as the infrastructure that provides utility services (such as water, gas, electricity and so forth) to buildings, residences and the like. The present disclosure provides for an automated identification and mapping of infrastructure and utility-related sites and features starting from ground-level and top- down imagery (top-down imagery can be considered as images taken from above, for example as aerial and / or satellite images). The process can locate and map these utility assets within vast geographic areas. Generative Al techniques infer parts of the infrastructure not explicitly represented in the raw input data (e.g., underground components). This can be used to enhance utility infrastructure management, maintenance, and planning.

[0035] This disclosure introduces an automated, data-driven solution to utility infrastructure mapping that can leverage advanced Computer Vision, Deep Learning and Generative Al techniques to extract valuable insights from ground-level and top-down imagery.

[0036] An apparatus and a method is provided for mapping both above-ground and belowground utility infrastructure.

[0037] FIG. 1 is a flow diagram of a computer-implemented method of utility infrastructure mapping. The steps of this method may be performed by the various modules of the utility infrastructure mapping apparatus 200 of FIG. 2.

[0038] The utility infrastructure mapping apparatus 200 comprises a ground-level image acquisition module 201 , an object detection module 202, a georeferencing module 203, a grid determination module 204, a top-down image acquisition module 205, a sensor fusion module 206, an above-ground map determination module 207, and a generative modelling module 208.

[0039] These modules may be realised as modules of code in a computer system. Whilst described as distinct modules, these modules can be embodied as a single module or groups of modules. For example, the module(s) can be realised as one or more processors executing instructions stored in computer storage. The operation of these modules is discussed in more detail with reference to FIG. 1.

[0040] Turning to FIG. 1, at step S101 , one or more ground-level images of a first geographic area are acquired.

[0041] In an example, step S101 can be performed by the ground-level image acquisition module 201 of the utility infrastructure mapping apparatus 200. The ground-level images can be captured using low-level cameras. In this sense, low- level or ground-level can be considered as being near to the ground, as opposed to top- down imagery.

[0042] For example, a low-level camera can be integrated into a vehicle that travels on the ground (e.g. as camera(s) integrated into / onto cars or vans or the like, or using dashcam footage, or a car-mounted cameras similar to Tesla Motors three mounted cameras).

[0043] Multiple low-level cameras can be used, for example as cameras integrated into multiple vehicles. Each camera can be considered as a data source for the ground-level images. These data sources can be calibrated and georeferenced to have consistent spatial and temporal alignment.

[0044] The ground-level images can be pre-processed to remove noise, correct for distortions, and / or to standardize formats.

[0045] The ground-level image acquisition module 201 can acquire the ground-level images from these sources, or from an intermediate datastore in which the captured groundlevel images are stored.

[0046] At step S102, object detection is performed on the ground-level images to identify one or more first utility infrastructure objects.

[0047] In an example, step S102 can be performed by the object detection module 202 of the utility infrastructure mapping apparatus 200.

[0048] The first utility infrastructure objects can include any objects that pertain to utility infrastructure that can be visible at ground level. These can be include electrical power lines, electrical substations, manholes covers (e.g., sewer manhole covers, stormwater drainage covers, utility access covers), and water and sewer infrastructure.

[0049] Object detection algorithms and deep learning models (such as YOLOv7, for example) can be utilised to segment the image and identify objects (such as manhole covers) visible from the low-level cameras. This is based on their distinct visual features.

[0050] In addition to object detection techniques, step S102 can also comprise utilising image segmentation techniques (such as Internimage and ONE-PEACE, for example) to localise and classify more objects of interest (such as manhole covers on roads) and thus combine and improve overall results.

[0051] At step S103, georeferencing can be applied to the ground-level images. In an example, step S103 can be performed by the georeferencing module 203 of the utility infrastructure mapping apparatus 200.

[0052] The ground-level images can be mapped to (approximate) geographical coordinates using GPS data, for example. For example, this can be brought about by recording GPS coordinates when capturing the images.

[0053] More precise mapping to geographical coordinates can then also subsequently occur by sensor fusion with top-down image data, as will be described with reference to step S109.

[0054] The captured images, along with data pertaining to the detected objects, and the georeferencing, can be stored in a datastore.

[0055] At step S104, a first grid is determined. The first grid represents the first geographic area. The first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images.

[0056] In an example, step S104 can be performed by the grid determination module 204 of the utility infrastructure mapping apparatus 200.

[0057] The first grid can be a two-dimensional grid wherein each grid point corresponds to a geographical point in the first geographical area.

[0058] The determined locations of the indications of the one or more first utility infrastructure objects on the first grid can be determined based on the georeferencing applied to the ground-level images.

[0059] Indications of the presence of utility infrastructure objects can be applied to the grid points corresponding to the objects determined locations, using a combination of the georeferenced images, and the identified first utility infrastructure objects.

[0060] When a utility infrastructure object (for example, a manhole cover) is identified, the indication of the presence of the identified object is applied to the grid in a position corresponding to the georeferenced location of the image.

[0061] In some examples, the indication of the presence of a utility infrastructure object can represent a probability score of such a utility infrastructure object being present at the location. This probability score can be based on a confidence in the identification of the object. In an example, this confidence can be a confidence score of an object classifier used to detect the objects. This can be the normalized level of activation of the output neurons that that represent the class of objects at hand.

[0062] At step S105, one or more top-down images of a second geographic area are acquired. The second geographic area is at least partly coincident with the first geographic area.

[0063] In an example, step S105 can be performed by the top-down image acquisition module 205 of the utility infrastructure mapping apparatus 200.

[0064] A top-down image can be considered as an image of the geographic area obtained at an elevation, for example as a substantially top-down view of the geographic area.

[0065] In some examples, top-down images can include images captured by satellite (including multispectral satellite imagery), images captured by drones / unmanned aerial vehicles (UAVs), and / or images captured by aircraft. The top-down images can be from more than one such source (e.g., a combination of drone / UAV footage with multispectral satellite imagery).

[0066] The top-down images can be pre-processed to remove noise, correct for distortions, and / or to standardize formats.

[0067] The top-down image acquisition module 205 can acquire the top-down images from these sources, or from an intermediate datastore in which the captured top-down images are stored.

[0068] The second geographic area, for which the top-down images are recorded, is at least partly coincident with the first geographic area for which the ground-level images are recorded. That is to say, the first geographic area and the second geographic area can be overlapping geographic areas. In this way, the top-down images and the groundlevel images can supplement one-another.

[0069] At step S106, object detection is performed on the top-down images to identify one or more second utility infrastructure objects.

[0070] In an example, step S106 can be performed by the object detection module 202 of the utility infrastructure mapping apparatus 200. In some examples, the object detection module that performs step S102 may be a different object detection module to that which performs step S106, or it may be the same module. In this step, object detection algorithms and deep learning models (such as YOLOv7) can be utilised to segment the images and identify utility infrastructure objects visible from a top-down perspective.

[0071] Second utility infrastructure objects can be considered as utility infrastructure objects that can be visible from an aerial and / or satellite view. Such objects can include, for example, power lines, electrical substations and other infrastructure components.

[0072] In addition to object detection techniques, image segmentation techniques (such as Internimage and ONE-PEACE) can be utilised to localise and classify more objects of interest.

[0073] At step S107, georeferencing can be applied to the top-down images.

[0074] In an example, step S107 can be performed by the georeferencing module 203 of the utility infrastructure mapping apparatus 200. In some examples, the georeferencing module that performs step S103 may be a different georeferencing module to that which performs step S107, or it may be the same module.

[0075] The internal coordinate system of the top-down photo can be related to a ground system of geographic coordinates. This can comprise identifying a plurality of ground control points with established locations on the image. Then, a curve fitting method can be deployed to produce a parametric formula to reference other points in the image, so that the rest of the image is georeferenced.

[0076] The captured images, along with data pertaining to the detected objects, and the georeferencing, can be stored in a datastore.

[0077] At step S108, a second grid is determined. The second grid represents the second geographic area. The second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images.

[0078] In an example, step S108 can be performed by the grid determination module 204 of the utility infrastructure mapping apparatus 200. In some examples, the grid determination module that performs step S104 may be a different grid determination module to that which performs step S108, or it may be the same module. In a similar manner to the first grid, the second grid can be a two-dimensional grid wherein each grid point in the second grid corresponds to a geographical point in the second geographical area.

[0079] The determined locations of the indications of the one or more second utility infrastructure objects on the second grid can be determined based on the georeferencing applied to the top-down images.

[0080] Indications of the presence of utility infrastructure objects can be applied to the grid points corresponding to the objects determined location, using a combination of the georeferenced images, and the identified second utility infrastructure objects.

[0081] When a utility infrastructure object (for example, a power line) is identified, the indication of the presence of the identified object is applied to the grid in a position corresponding to the georeferenced location of the image.

[0082] In some examples, the indication of the presence of a utility infrastructure object can represent a probability score of such a utility infrastructure object being present at the location. This probability score can be based on a confidence in the identification of the object. In an example, this confidence can be a confidence score of an object classifier used to detect the objects. This can be the normalized level of activation of the output neurons that that represent the class of objects at hand.

[0083] At step S109, sensor fusion is performed to determine an integrated grid by integrating the first grid and the second grid.

[0084] In an example, step S109 can be performed by the sensor fusion module 206 of the utility infrastructure mapping apparatus 200.

[0085] The inputs for step S109 can comprise the first grid and the second gird. Sensor fusion techniques (for example, a Kalman Filter) can be used to integrate the two grids, and determine the integrated (i.e. combined) grid that corresponds to the combination of the first grid and the second grid. This can improve the accuracy of utility infrastructure object detection.

[0086] Step S109 can further comprise using, from the top-down images, thermal and multispectral satellite imagery, which captures data across multiple spectral bands, to detect anomalies in electromagnetic (EMG) fields and / or utilise additional 2D visual information that is hidden in standard RGB (red-green-blue) images. Step S109 can further comprise using, from the top-down images, spectral indices, such as Normalized Difference Vegetation Index (NDVI), to highlight areas with potential EMG field disturbances, such as changes in vegetation health that can indicate underground cable faults or EMG field variations.

[0087] Thermal imagery can also be incorporated into step S109, to detect temperature anomalies associated with EMG fields. Overheating or abnormal temperature patterns can indicate issues in electrical components.

[0088] In some examples, the integrated grid can be combined with data from EMG sensors that measure electromagnetic fields in the vicinity of power lines and substations. These sensor readings can be correlated with the visual data to identify areas with EMG field disturbances or deviations from expected levels.

[0089] At step S110, an above-ground utility infrastructure map is determined. The aboveground utility infrastructure map comprises locations of utility infrastructure objects based on the integrated grid.

[0090] The above-ground utility infrastructure map can also include the type of identified utility infrastructure object at the locations (e.g., manhole cover, substation, power line, etc). This can be based on labels for these objects, as determined during the object detection at steps S102 and S106.

[0091] In an example, step S110 can be performed by the above-ground map determination module 207 of the utility infrastructure mapping apparatus 200.

[0092] The combination of the first grid and the second grid, as the integrated grid, can represent an above-ground utility infrastructure map. This map can indicate the locations of the identified utility infrastructure objects, that are located above ground, from both the ground-level images and the top-down images.

[0093] At step S111 , generative modelling is performed to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the aboveground utility infrastructure map.

[0094] The locations of utility infrastructure objects, and their type, can be used to map corresponding underground utility infrastructure.

[0095] In an example, step S111 can be performed by the generative modelling module 208 of the utility infrastructure mapping apparatus 200. The output of step S110 is an above-ground map of utility infrastructure (e.g. manhole covers). However, much utility infrastructure lies underground and is therefore not visible to the ground-level and top-down imagery.

[0096] Therefore, at step S111 , generative modelling can be used to infer the distribution of underground utility infrastructure. This can produce an accurate digital map overlay with identified utility-related objects and their precise geospatial coordinates.

[0097] The underground utility infrastructure map can be considered as a map of utility infrastructure objects that are underground. Underground utility infrastructure can be considered as utility infrastructure such as underground electrical power lines, underground electrical substations, water infrastructure, sewer infrastructure, and the like.

[0098] To create the underground utility infrastructure map containing the locations of underground utility infrastructure objects for a particular geographical area, various inputs can be given to Generative Al models (based on I similar to StyleGAN-XL, StyleNAT and GANformer2). Such models can have, as inputs, both textual and visual information, and generate the underground utility infrastructure map as an image.

[0099] In an example, the input information for the generative modelling at step S111 can include a model of the geographic area that is based on the integrated grids of step S109 from the fusion of the ground-level and top-down images. The input can further comprise structured textual data of the identified objects from the output of the object detection and segmentation models.

[0100] An additional input to the generative modelling can include information on distinctions between different building types in the geographic area. The type of the building can be automatically inferred from the top-down images (such as the aerial images). Alternatively or additionally, the apparatus can receive explicit information about the type of buildings, for example as a textual input. The generative modelling can use information on the building height as an input to improve predictions on the type and density of utilities connecting to the building. For example, this can differ between a detached single-family residential building, and a high-rise apartment or office building. The visual representation of buildings can automatically be picked by the system as relevant features in the learning process. For example, tall buildings may be correlated through the learning process to the existence of underground electrical infrastructure (substations). Similarly, the visual representation buildings in residential areas with family homes can be associated through the learning to underground infrastructure specific to that type of neighbourhood.

[0101] The aforementioned methodology does not rely on pre-existing underground plans or map data. Instead, this underground mapping is generated using the inferential algorithms.

[0102] The method and apparatus of the present disclosure can provide many benefits and advantages. One advantage can include improved maintenance and repairs of utility infrastructure through accurate mapping that allows for proactive maintenance and faster repairs of infrastructure, thereby reducing downtime and service disruptions. Another advantage can include improved project planning in the avoidance of potential utility conflicts during design, i.e. before a shovel touched the ground. Furthermore, reduced costs associated with unnecessary utility relocations, avoidable construction delays, and contractor change orders can be achieved. As will as this, tighter contractor bid estimates can be realised by providing a more accurate design. Also, improved urban planning can be realised to ensure new developments do not disrupt existing utility networks. Another advantage can be in disaster recovery. The mapping can assist in post-disaster recovery efforts, enabling utilities to restore services quickly. Another advantage can be in data-driven decision-making in that the infrastructure mapping can provide data for informed decision-making, helping utilities plan for expansion and modernization.

[0103] In some examples, the methodology can further comprise fine-tuning the generative model of step S111 based on prospective data. As additional labelled data is collected, this can be used to update the generative model. In other words, as additional data is gathered over time, the generative model improves its performance. For example, if a construction team digs up a section of road and discovers that the underground utility equipment corresponds to an earlier building standard (e.g. relating to a particular depth, spacing or type of underground infrastructure), the self-learning algorithm updates local predictions to align with this new information. Based on data about the age of a building or road, this self-learning model uses pattern matching to infer the likely distribution of utility infrastructure, without the human user of the system explicitly specifying such constraints. The request for generation can be updated using free text with a high degree of transparency to non-technical end users. The fine-tuning of the generative model can be brought about through a fine-tuning technique, such as low-rank adaptation, by constraining the rank of an update matrix based on rank decomposition.

[0104] In some examples, the apparatus can be configured to output a warning to a user. Such an example can include a user interface that enables visualization and analysis of identified utility-related sites. An alert can be generated and output, for an operator of the apparatus, when there is a risk of damaging existing underground infrastructure, based on the above-ground utility infrastructure map and the underground utility infrastructure map, and the determined locations and types of utility infrastructure objects. For example, this could be during the planning and / or implementation of infrastructure projects. The system can generate reports and alerts for maintenance and inspection based on the detected changes. Such alerts may include visual and / or audible alerts output from the apparatus.

[0105] FIG. 3 depicts a high-level block diagram of an apparatus 300 suitable for implementing various aspects of the disclosure. Although illustrated in a single block, in other embodiments the apparatus 300 may also be implemented using parallel and distributed architectures. Thus, for example, various steps such as those illustrated in the methods described above by reference to FIG. 1 may be executed using apparatus 300 sequentially, in parallel, or in a different order based on particular implementations. The apparatus 110 of FIG. 2 can be implemented in the form of apparatus 300.

[0106] According to an example, depicted in FIG. 3, apparatus 300 comprises a printed circuit board 301 on which a communication bus 302 connects a processor 303 (e.g., a central processing unit "CPU"), a random access memory 304, a storage medium 311 , possibly an interface 305 for connecting a display 306, a series of connectors 307 for connecting user interface devices or modules such as a mouse or trackpad 308 and a keyboard 304, a wireless network interface 310 and / or a wired network interface 312. Depending on the functionality required, the apparatus may implement only part of the above. Certain modules of FIG. 3 may be internal or connected externally, in which case they do not necessarily form integral part of the apparatus itself. E.g. display 306 may be a display that is connected to the apparatus only under specific circumstances, or the apparatus may be controlled through another device with a display, i.e. no specific display 306 and interface 305 are required for such an apparatus.

[0107] Memory 311 contains software code which, when executed by processor 303, causes the apparatus to perform the methods described herein. In an example, a detachable storage medium 313 such as a USB stick may also be connected. For example the detachable storage medium 313 can hold the software code to be uploaded to memory 311.

[0108] The processor 303 may be any type of processor such as a general purpose central processing unit ("CPU") or a dedicated microprocessor such as an embedded microcontroller or a digital signal processor ("DSP").

[0109] In addition, apparatus 300 may also include other components typically found in computing systems, such as an operating system, queue managers, device drivers, or one or more network protocols that are stored in memory 311 and executed by the processor 303.

[0110] Although aspects herein have been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure. It is therefore to be understood that numerous modifications can be made to the illustrative embodiments and that other arrangements can be devised without departing from the spirit and scope of the disclosure as determined based upon the claims and any equivalents thereof.

[0111] For example, the data disclosed herein may be stored in various types of data structures which may be accessed and manipulated by a programmable processor (e.g., CPU or FPGA) that is implemented using software, hardware, or combination thereof.

[0112] It should be appreciated by those skilled in the art that block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that flow charts, flow diagrams, state transition diagrams, and the like represent various processes which may be substantially implemented by circuitry.

[0113] Each described function, engine, block, step can be implemented in hardware, software, firmware, middleware, microcode, or any suitable combination thereof. If implemented in software, the functions, engines, blocks of the block diagrams and / or flowchart illustrations can be implemented by computer program instructions I software code, which may be stored or transmitted over a computer-readable medium, or loaded onto a general purpose computer, special purpose computer or other programmable processing apparatus and I or system to produce a machine, such that the computer program instructions or software code which execute on the computer or other programmable processing apparatus, create the means for implementing the functions described herein. In the present description, block denoted as "means configured to perform ..." (a certain function) shall be understood as functional blocks comprising circuitry that is adapted for performing or configured to perform a certain function. A means being configured to perform a certain function does, hence, not imply that such means necessarily is performing said function (at a given time instant). Moreover, any entity described herein as "means", may correspond to or be implemented as "one or more modules", "one or more devices", "one or more units", etc. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional or custom, may also be included. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

Claims

Claims1. A computer-implemented method of utility infrastructure mapping, the method comprising: acquiring one or more ground-level images of a first geographic area; performing object detection on the ground-level images to identify one or more first utility infrastructure objects; determining a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; acquiring one or more top-down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; performing object detection on the top-down images to identify one or more second utility infrastructure objects; determining a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; performing sensor fusion to determine an integrated grid by integrating the first grid and the second grid; determining an above-ground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and performing generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the aboveground utility infrastructure map.

2. The computer-implemented method of claim 1 , wherein the ground-level images are captured using one or more low-level cameras, and / or wherein the top-down images are aerial images and / or satellite images.

3. The computer-implemented method of any preceding claim, wherein the first utility infrastructure objects comprise objects that pertain to utility infrastructure that is visible at ground level, and / or wherein the second utility infrastructure objects comprise objects that pertain to utility infrastructure that is visible from an aerial and / or satellite view.

4. The computer-implemented method of any preceding claim, wherein the performing object detection on the ground-level images further comprises performing image segmentation to localise and classify one or more first utility infrastructure objects; and / or wherein performing object detection on the top-down images further comprises performing image segmentation to localise and classify one or more second utility infrastructure objects.

5. The computer-implemented method of any preceding claim, wherein the indications of the one or more first utility infrastructure objects at locations on the first grid represent a probability score of such a utility infrastructure object being present at the location; and / or wherein the indications of the one or more second utility infrastructure objects at locations on the second grid represent a probability score of such a utility infrastructure object being present at the location.

6. The computer-implemented method of any preceding claim, wherein the method further comprises: applying georeferencing to the ground-level images, wherein the determined locations of the indications of the one or more first utility infrastructure objects on the first grid are determined based on the georeferencing applied to the ground-level images; and applying georeferencing to the top-down images, wherein the determined locations of the indications of the one or more second utility infrastructure objects on the second grid are determined based on the georeferencing applied to the top-down images.

7. The computer-implemented method of claim 6, wherein applying georeferencing to the ground-level images comprises mapping the ground-level images to geographical coordinates; and / or wherein applying georeferencing to the top-down images comprises identifying a plurality of ground control points with established locations on the top-down image, and deploying a curve fit to produce a parametric formula referencing other points in the top- down image.

8. The computer-implemented method of any preceding claim, wherein the method further comprises: pre-processing the ground-level images to remove noise, correct distortions and / or standardize formats; and / or pre-processing the top-down images to remove noise, correct distortions and / or standardize formats.

9. The computer-implemented method of any preceding claim, wherein the top-down images comprise thermal and / or multispectral satellite imagery, and the method further comprises detecting anomalies in electromagnetic fields, and / or utilising visual information hidden in red-green-blue images, based upon the thermal and / or multispectral satellite imagery.

10. The computer-implemented method of any preceding claim, wherein the method further comprises applying Normalized Difference Vegetation Index analysis to the top- down images to determine areas with potential electromagnetic field disturbances based on changes in vegetation health identified from the Normalized Difference Vegetation Index analysis.

11. The computer-implemented method of any preceding claim, wherein the top-down images comprise thermal imagery, and the method further comprises detecting temperature anomalies associated with electromagnetic fields.

12. The computer-implemented method of any preceding claim, wherein the method further comprises combining the integrated grid with data from electromagnetic field sensors that measure electromagnetic fields in vicinities around power lines and / or electrical substations, and correlating the data from the electromagnetic field sensors with data from the ground-level images and / or top-down images to identify areas with electromagnetic field disturbances or deviations from expected levels.

13. The computer-implemented method of any preceding claim, wherein the method further comprises outputting an alert when, based on the above-ground utility infrastructure map and underground utility infrastructure map, and the determined locations of utility infrastructure objects, there is a risk of damaging existing underground utility infrastructure.

14. A utility infrastructure mapping apparatus comprising: a ground-level image acquisition module configured to acquire one or more ground-level images of a first geographic area; an object detection module configured to perform object detection on the groundlevel images to identify one or more first utility infrastructure objects; a grid determination module configured to determine a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; a top-down image acquisition module configured to acquire one or more top- down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; an object detection module configured to perform object detection on the top- down images to identify one or more second utility infrastructure objects; a grid determination module configured to determine a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; a sensor fusion module configured to perform sensor fusion to determine an integrated grid by integrating the first grid and the second grid; an above-ground map determination module configured to determine an aboveground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and a generative modelling module configured to perform generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the above-ground utility infrastructure map.

15. A computer-readable medium storing instructions thereon, that when executed by one or more processors, cause the one or more processors to perform the following steps: acquiring one or more ground-level images of a first geographic area; performing object detection on the ground-level images to identify one or more first utility infrastructure objects; determining a first grid representing the first geographic area, wherein the first grid comprises indications of the one or more first utility infrastructure objects at locations on the first grid corresponding to determined locations from the ground-level images; acquiring one or more top-down images of a second geographic area, wherein the second geographic area is at least partly coincident with the first geographic area; performing object detection on the top-down images to identify one or more second utility infrastructure objects; determining a second grid representing the second geographic area, wherein the second grid comprises indications of the one or more second utility infrastructure objects at locations on the second grid corresponding to determined locations from the top-down images; performing sensor fusion to determine an integrated grid by integrating the first grid and the second grid; determining an above-ground utility infrastructure map comprising locations of utility infrastructure objects based on the integrated grid; and performing generative modelling to determine an underground utility infrastructure map based on the locations of the utility infrastructure objects in the aboveground utility infrastructure map.

Images