Method and system for accessing two-dimensional drawing and method for adding geo-reference data

Abstract

A system and method for accessing remotely located two-dimensional (2D) mapping by a mobile device at a worksite includes a reference device at the worksite configured to provide a geographic reference to the mobile device. A document retrieval system is configured to access a plurality of 2D drawings in a document library. The mobile device provides the annotated document to a document retrieval system. The document retrieval system accesses the 2D drawing in the document library based on the annotated document and provides a rasterized version of the 2D drawing to the mobile device.

Description

Technical Field

[0001] The present invention relates to document retrieval systems and, more particularly, to remote access to geographically tagged two-dimensional CAD drawings. Background Art

[0002] In the business-to-business (B2B) sector, workers may need two-dimensional (2D) drawings to facilitate or even perform work-related tasks. The main B2B sectors include:

[0003] - Architecture, engineering and construction (typically involving documentation for assets such as buildings, structures and infrastructure),

[0004] - cities, regions and public services (where 2D mapping can be targeted at, for example, administrative districts, departments, states, counties and cities),

[0005] - Large assets (where 2D mapping can be relevant to assets such as marine and offshore, chemical, nuclear, utilities, etc.), and

[0006] - Manufacturing (wherein the 2D drawings may be related to, for example, discrete, batch and / or continuous manufacturing).

[0007] 2D drawings are also used to cover other B2B areas of activity for human work, as well as business-to-consumer (B2C) or business-to-business-to-consumer (B2B2C) involving the public, in particular interior design (e.g. HomeByMe).

[0008] 2D drawings used in these areas (and others) can cover the cradle-to-grave life cycle of a work site and the reclamation of land. The use of these 2D drawings is often collaborative, but can also be individual.

[0009] Work tasks in the areas mentioned above may involve some team members working in an office (usually away from the field), and other team members working in the field.

[0010] Typically, only a few team members have a global view of the task of working with 3D CAD models, 2D drawings, and 2D rasterized drawings. Current software tools usually address their needs, and their view of the project emphasizes collective operations.

[0011] In contrast, most work by workers in the field covers only a given task or process at a time, and in most cases is supported only by 2D vector drawings (2D drawings in vector graphics format that inherently do not include geographic coordinates) and 2D rasterized drawings. Several workers can work on the same process at the same time, performing their tasks until the task is completed. When several workers perform the same task, their tasks are performed in parallel, but not necessarily synchronously.

[0012] Workers often need to access relevant 2D rasterized drawings to complete their work tasks. For example, workers can:

[0013] ● Want to reference relevant 2D rasterized drawings to complete their tasks,

[0014] ● Encountering problems with the completion status of many different causes or tasks,

[0015] ●Want to associate each issue and / or completion status and communicate it to the appropriate team,

[0016] ● Propose solutions to problems, and / or

[0017] ●If possible, try to solve the problem on the spot.

[0018] Effective management and optimization of its business processes (each process is a set of tasks defined in a graphical representation diagram such as BPMN), as well as monitoring of the execution and interdependencies of the various business processes can impact the success of a project.

[0019] Better tools are needed to increase productivity through the timely use of 2D drawings and 2D rasterized drawings provided between workers and related document management in the field and remote offices. Currently, the overall view can problematically obscure the importance of the individual tasks that make up the business process and their management.

[0020] Most of the work done in the B2B space mentioned above involves 2D drawings and 2D rasterized drawings. Most assets have 100s, 1000s, or even more 2D drawings and 2D rasterized drawings to be used on site. Many designers still use 3D modeling, primarily for the purpose of creating 2D drawings.

[0021] The tools currently available to workers in the field can hinder overall productivity in the areas mentioned above, causing productivity to lag far behind other industries, especially manufacturing. For example, in the US construction industry, the value added per full-time equivalent worker in 2019 was the same as the value added per full-time equivalent worker in 1950 (Becker Friedman Institute). In recent decades, there have been few significant advances in the common field 2D drawing tools available to construction workers.

[0022] While document retrieval has been improved by using libraries to create a single source of truth and to support collaborative work through codified processes such as check-in and check-out, users still have to gain access to that single library, apply filters and search for keywords within the library (or write queries, as SQL) to retrieve relevant data for a document, which may include a 2D drawing or a 2D rasterized drawing.

[0023] While consumer devices can be used to locate other objects or determine their own location, the inaccuracies of consumer devices make them unreliable for pinpointing true field locations for professional work.

[0024] 2D drawings use vector graphics formats that inherently do not include geographic coordinates. For example, in a drawing in the DWG format, the coordinate system used is a right-handed Cartesian coordinate system, where the X-axis represents the horizontal direction, the Y-axis represents the vertical direction, and the Z-axis represents the depth or height. There is no built-in structure for storing geographic coordinates (latitude, longitude, and altitude). Other systems, such as geographic information systems (GIS), rely on separate representations of 2D drawings in different formats.

[0025] Current software for including geographic coordinates in 2D drawing formats has great limitations. For example, some schemes make it possible to add geographic coordinates to some selected reference points of a particular 2D drawing. However, such 2D drawings typically represent an overall site plan or layout of an intended construction, but are not represented in other 2D drawings. The selected reference points correspond to geodetic points of specific geographic locations, such as the corners or boundaries of a unified parcel, and some points on the periphery of an asset. The licensing process for any asset on site requires reference to these geodetic points. The precise location of the geodetic points is measured on site by professional surveyors using a measurement system (usually a Universal Transverse Mercator (UTM)).

[0026] Office staff spend a lot of time searching for relevant 2D drawings and 2D rasterized drawings that have been converted into digital form. As an example, office staff of French municipalities usually spend an average of 90 minutes a day searching for relevant 2D drawings and 2D rasterized drawings.

[0027] Although workers in the field can access relevant 2D rasterized drawings at the beginning of a planned task, the scheme of assigning 2D rasterized drawings to each worker's task generally does not take into account problems that occur during the performance of field work, such as reassignment of workers, delays due to supply chain issues, quality of materials or equipment, or removal of previous work. When a worker does not have the relevant 2D rasterized drawing, the worker's retrieval of the relevant 2D rasterized drawing requires the worker to have access to a mobile device and have sufficient information and vocabulary knowledge to access a library of 2D drawings using the mobile device, select filters related to the worker's current work, and search the library using keywords to obtain the relevant 2D drawings. Alternatively, the worker must request help from colleagues in the office on site or by calling, or request help through a helpline to find the relevant 2D drawings, which are usually in rasterized form.

[0028] Most reference devices for indoor positioning systems (IPS) are very accurate when locating smartphones at small distances or in direct line of sight, but are not very accurate under normal indoor use conditions within an asset. The coverage of the reference devices is limited to the area where location information can be obtained from the mobile device. As the coverage area increases, the accuracy decreases. Some expensive systems that require antennas, readers, tags, greater power requirements, and configuration and installation can have better accuracy for a given use case.

[0029] When a 3D model is used for the design of an asset, the 3D design model does not always reflect the current state of the asset as built or repaired, because the available 2D drawings of the as built or repaired asset may not have been updated. Getting information from the design 3D model does not guarantee correctness unless the design 3D model is a perfect digital twin that is maintained throughout its lifecycle.

[0030] Some applications refer to Building Information Models (BIM models), such as Dalux Field software. Here, document retrieval requires access to libraries, definition of filters, and definition of keywords for queries. Unfortunately, the scope of use of these applications is limited to new assets. Many assets are not built using BIM models, and even today, many new assets do not use BIM as a design method. When there is no BIM model available, only 2D drawings and 2D rasterized drawings are available, which these applications do not cover.

[0031] Therefore, there is a need in the industry to address one or more of the shortcomings mentioned above. Summary of the invention

[0032] Embodiments of the present invention provide a system and method for retrieving two-dimensional drawings. Briefly described, the present invention is directed to a system and method for accessing a remotely located two-dimensional (2D) drawing via a mobile device at a work site. A reference device at the work site is configured to provide a geographic reference to the mobile device. A document retrieval system is configured to access a plurality of 2D drawings in a document repository. The mobile device provides annotated documents to the document retrieval system. The document retrieval system accesses the 2D drawing in the document repository based on the annotated documents and provides a rasterized version of the 2D drawing to the mobile device.

[0033] By examining the following drawings and detailed description, other systems, methods and features of the present invention will be or become apparent to one of ordinary skill in the art. It is intended that all such additional systems, methods and features be included within this description, within the scope of the present invention and protected by the accompanying claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are included to provide a further understanding of the present invention, and are incorporated in and constitute a part of this specification. The components in the drawings are not necessarily drawn to scale, but emphasis is placed on clearly illustrating the principles of the present invention. The accompanying drawings illustrate embodiments of the present invention and together with the description serve to explain the principles of the present invention.

[0035] Figure 1 It is a schematic diagram of a system block diagram of a first embodiment of a document retrieval system.

[0036] Figure 2 is a flow chart of an exemplary method embodiment for using a document retrieval system.

[0037] Figure 3 is a data structure diagram of a data structure of an exemplary embodiment of a 2D drawing lookup table for a document retrieval system.

[0038] Figure 4 is a flow chart of an exemplary method for adding geo-referenced data to a document.

[0039] Figure 5 is a schematic diagram showing an example of a system for performing the functions of the present invention.

[0040] Figure 6 is a chart comparing the maximum geographic area covered by a given paper size format at a given scale to the current horizontal error of the measurement. DETAILED DESCRIPTION

[0041] The following definitions are used to explain terms applied to features of the embodiments disclosed herein and are only meant to define elements within the present disclosure.

[0042] As used in this disclosure, "asset" may refer to a geographically locatable resource of economic value owned or controlled by an individual, company, or country, such as a land area, a plot of land, a park, a building, a structural part of an infrastructure, and / or a plant (e.g., a chemical, nuclear, oil and gas plant), etc.

[0043] As used in this disclosure, "site" refers to the geographic location of an asset.

[0044] As used in this disclosure, a "worker" is a person who works on site. The work can be planning, auditing, construction, renovation, manufacturing, interior design, or any other type of corporate or contract work. Many work tasks involve the use of two-dimensional (2D) drawings for each of the following task components: planning, organizing, executing, monitoring, and controlling.

[0045] As used in this disclosure, a "mobile device" is a processor-based device that can be easily moved by a worker, such as a smartphone, tablet, or laptop. A mobile device is capable of:

[0046] ● taking photos and / or videos and / or images (e.g., LIDAR scans) from its location,

[0047] ● Associate annotations, metadata (including geolocation) and / or voice recordings and sounds,

[0048] ● share the above items digitally with selected others, and

[0049] ●When connected, receive data from the library system, the data includes 2D drawings and rasterized 2D drawings.

[0050] As used in this disclosure, a "document retrieval system" refers to a computer-based device or a group of communication or connection devices configured to receive data as input and use the input data to locate and access documents stored in a library system. The input data received by the document retrieval system may include one or more of keywords, specifications, images, written descriptions, document identifiers, document indexes, coordinates, and / or optical codes (e.g., bar codes or QR codes).

[0051] As used in this disclosure, a "library system" or "document library system" refers to a device or group of devices for storing, organizing, and accessing 2D drawings and rasterized 2D drawings. A library system may include one document storage device or multiple document storage devices, which may be co-located or geographically distributed, such as accessible via a communication network.

[0052] As used in this disclosure, a "reference device" is a device (usually small) placed on site at a reference point with known coordinates. The reference device transmits its relative position relative to the mobile device so that when a photo, video or image is taken by the mobile device, the position of the photo, video or image can be offset by the relative position relative to the reference device to calculate the geographic coordinates of the mobile device. The reference device can be based on one or more of several technologies such as radio frequency, ultrasound and infrared. In most applications, the preferred technology is radio frequency. Within the radio frequency range, there may be many types of communications between the reference device and the mobile device, such as but not limited to ultra-wideband (UWB), chirp spread spectrum (CSS), LTE transmission, Bluetooth (including low power transmission (BLE)), Wi-Fi, cellular, time difference of arrival (TDoA) and two-way ranging (TWR).

[0053] As used in this disclosure, unless otherwise specified as a "2D rasterized drawing", a "2D drawing" uses a vector graphics format that uses geometric primitives (e.g., points, lines, curves) to store CAD entities and their attributes (e.g., dimensions, layers, and annotations). The format is scalable and editable. Examples of 2D drawing formats include DXF, DWG, DWF, SVG, and Adobe Illustrator, which are sometimes referred to as "vector" drawings.

[0054] As used in this disclosure, the "2D rasterized drawing" format uses a resolution-dependent rasterized image to display pixelation at high resolution. The capabilities of the 2D rasterized drawing are limited by certain edits, such as cropping and contrast settings, but can be annotated. For the same view from the user, the 2D rasterized drawing file size is a more compact format than the vector graphics file size from which the 2D rasterized drawing file is derived.

[0055] The detailed description involves several systems for measuring position, including:

[0056] ○Survey-grade GPS / GNSS

[0057] Survey-grade GPS / GNSS receivers that utilize differential correction techniques, such as real-time kinematics (RTK) or post-processed kinematics (PPK), can achieve much higher accuracy than consumer-grade receivers. These techniques involve using base stations with known positions or post-processed data to refine the measurement and improve accuracy.

[0058] Typically, under favorable conditions, RTK surveys can achieve accuracies within 1-2 cm horizontally and 2-3 cm vertically. PPK surveys, which involve collecting data in the field and then processing the data using a base station or known reference points, can provide even higher accuracy, potentially reaching sub-centimeter accuracy.

[0059] ○Consumer-grade GPS

[0060] Typically, consumer mobile devices typically have much less accurate GPS receivers than dedicated survey-grade GPS / GNSS receivers. The accuracy of GPS measurements from consumer devices can range from a few meters to tens of meters.

[0061] Smartphones can also supplement GPS information with alternative positioning technologies such as Wi-Fi positioning and cellular network data when GPS signals are weak or unavailable.

[0062] ○Differential GPS (DGPS) and RTK services

[0063] Differential GPS (DGPS) and RTK services improve the accuracy of GPS. These services can send correction data from the known location and distance of a fixed ground-based reference station to one or more satellites in a satellite constellation. The spacing of the reference stations and the number of satellites used depends on the required accuracy and redundancy of coverage. Such systems can be used for maritime navigation, surveying, and agriculture. Currently, Starlink currently has 6,100 satellites.

[0064] ○Wi-Fi positioning

[0065] Wi-Fi positioning is also known as Wi-Fi based positioning or Wi-Fi fingerprinting, which utilizes Wi-Fi networks in the surrounding area. When Wi-Fi is enabled on a smartphone, the smartphone continuously scans for nearby Wi-Fi access points. The smartphone compares the identifiers (MAC addresses) of these Wi-Fi networks with a database that associates known Wi-Fi access points with their geographic locations.

[0066] By matching detected Wi-Fi access points with their known locations, a smartphone can estimate its location based on the proximity of these access points. This method is particularly useful indoors or in urban areas where GPS signals can be blocked or attenuated. However, this method does not have high accuracy within a range of a few meters to tens of meters or more.

[0067] ○Cellular data

[0068] Smartphones can also use cellular network data to determine location. When a smartphone is connected to a cellular network, the smartphone is constantly communicating with nearby cell towers. Each cell tower covers a specific geographic area, and the smartphone can estimate its location based on the signal strength and timing of the signals received from different towers. This method, called cell tower triangulation or network-based positioning, relies on knowledge of cell tower locations and signal propagation characteristics. The smartphone compares the signal strength and timing from multiple cell towers to estimate its approximate location. Similar to Wi-Fi positioning, cellular network data is often used when GPS signals are weak or unavailable, but cellular network data provides a less accurate position fix than GPS.

[0069] Now, reference will be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0070] As mentioned in the Background section, different areas rely on documentation for different reasons. For example, in a typical Architecture, Engineering, Construction (AEC) project, less than 15% of the team members work on the project as a whole. The staff includes construction project managers, construction designers, and construction model coordinators who manage the project, implement business processes for team members to follow, improve business processes during the project, and monitor the overall progress of the project and the associated costs and budgets. Their tasks include, for example:

[0071] ● planning and construction of a building, structure or part of an infrastructure,

[0072] ● refurbishment of a building, structure or part of an infrastructure,

[0073] ● auditing a building, structure, or part of an infrastructure,

[0074] ●Maintenance of a building, structure, or part of an infrastructure, and

[0075] ● Demolition of a building, structure, or part of an infrastructure.

[0076] Typically, 85% of the team members are staff who perform tasks on site.

[0077] For cities, regions and public services, regional engineering is often based in the office. Staff in the field need to communicate with their office. Some of these staff may be working on contract tasks outsourced to their own organization. For example, their tasks may include:

[0078] ● planning and construction of structural parts of buildings or infrastructure,

[0079] ●Parks and gardens management,

[0080] ● Refurbishment of structural parts of buildings or infrastructure,

[0081] ● Auditing, commissioning or health monitoring of structural parts of buildings or infrastructure,

[0082] ● Energy performance improvements to structural parts of buildings or infrastructure,

[0083] ●Maintenance of structural parts of buildings or infrastructure, and

[0084] ●Demolition of structural parts of buildings or infrastructure.

[0085] For large asset areas, workers in the field often need to communicate with their office. For example, their tasks may include:

[0086] ● Detailed planning and construction of assets,

[0087] ○ Debugging assets,

[0088] ●Maintain assets,

[0089] ○ Detect problems through daily duties or operational events,

[0090] ○ Identify devices and components,

[0091] ○ Access the relevant 2D drawing,

[0092] ■ The skill level of the maintenance operator, and

[0093] ■Safety Tips,

[0094] ●Retirement procedures,

[0095] ● Removal and access to installation manuals, and

[0096] ●Re-debug the program.

[0097] For the manufacturing field, workers in the field may need to communicate with their office. Manufacturing business processes may include, for example:

[0098] ● Discrete manufacturing,

[0099] ●Batch manufacturing, and

[0100] ●Continuous manufacturing.

[0101] The exemplary embodiments of the present invention establish a retrieval system for 2D drawings and 2D rasterized drawings, facilitate the use of the retrieval system, and make the retrieval system traceable and auditable. The retrieval system greatly improves the productivity of workers and the flexibility of workers to complete correctly, providing efficiency to the associated business processes.

[0102] Embodiments prepare libraries to include reference points with geographic coordinates on each 2D drawing of an asset, for example to reuse reference points in submitted 2D drawings, such as site plans and site layouts, during the licensing phase. In addition, detailed satellite views become more accurate. For example, IGN (National Institute of Geographic and Forest Information) publishes detailed maps by using LiDAR to collect horizontal and vertical data. For large cities in France, this data corresponds to 10 points per square meter.

[0103] Typically, each 2D drawing has its own scale. The scales typically vary within a wide range, e.g., for site plans, the scale is typically 1 / 1000; for layout plans, the scale is typically 1 / 500; for more detailed plans, the scales are typically 1 / 100 and 1 / 50. 2D drawings for installation details are typically not geo-located, since the same installation may involve several installations in adjacent drawings. An embodiment addresses this problem in the following manner.

[0104] To facilitate the reference point for generating each 2D plot, it is desirable to start with the largest scale plot and proceed to 2D plots with progressively smaller scales.

[0105] An embodiment starts with a geodetic point and derives geographic coordinates (North, South, East, West) in combination with scale. Enclosing the 2D drawing using a quadrilateral space oriented along the coordinates (N, S, E, W) enables an embodiment to find the geographic coordinates of the four corners of each 2D drawing comprising the plan. For multi-layer 2D drawings (e.g., ordinary 2D drawings), an embodiment separates the layers by type of work in the original format.

[0106] For 2D drawings that do not contain floor plans but do contain data that references a 2D drawing (for example, annotations, schedules, legends, and details), these 2D drawings are identical to the referenced 2D drawing. Figure 1 The 2D drawings are grouped together into packages. It is also possible to generate a rasterized version of each individual 2D drawing or package of 2D drawings from the 2D drawings. For example, the identifier of each 2D drawing and 2D rasterized drawing and its type of work are entered on a line, followed by the geographic coordinates of the 4 corners of the 2D drawing and 2D rasterized drawing. The line for each 2D drawing is used to create a lookup table. The lookup table can then be used to associate the provided geographic coordinates with the relevant 2D drawing and 2D rasterized drawing.

[0107] Out of Figure 1For illustrative purposes shown, embodiments are disclosed in the context of an exemplary scenario 100 in which a worker 112 located at a work site 120 uses a document retrieval system 150 to access relevant 2D drawings stored in a library 165 having one or more document storages 160. Here, the worker 112 seeks to access 2D drawings or 2D rasterized drawings related to a type of work being performed (e.g., construction, electrical and mechanical, fire safety, etc.). The worker 112 uses a mobile device 110 (e.g., a computer-based smartphone, tablet computer, or laptop computer) to take a photo (e.g., a selfie) at the site location and sends the photo to the document retrieval system 150, for example, over a network (e.g., the Internet or a cellular communication network). If the mobile device 110 has unimpeded global positioning system (GPS) signal reception, the mobile device 110 collects metadata of the geographic coordinates of the mobile device 110 and provides the geographic coordinates to the document retrieval system 150.

[0108] If GPS signal reception is impeded, geographic coordinates can be provided by a reference device 115 located at the site 120. The reference device 115 can be, for example, an indoor positioning system (IPS) device using radio frequency, such as Apple's AirTag, or a device manufactured by another vendor, such as Estimote, Kontakt.io, or Gimbal. The reference device 115 is placed at a selected reference point with known geographic coordinates (including altitude). The selected position and altitude of the reference device are added to the relevant 2D drawing (in this example, a photograph) and sent to the document retrieval system 150.

[0109] The document retrieval system 150 receives metadata from the mobile device 110 of the worker 112, which contains an identifier of the reference device 115 used and an offset from the reference device 115. The offset can be presented in different ways, such as as a delta expressed in coordinates, or as a distance and direction between the reference device 115 and the mobile device 110. The document retrieval system 150 calculates the position of the mobile device 110 in UTM coordinates and identifies the relevant 2D drawing, for example, using a library lookup table.

[0110] The document retrieval system 150 identifies the location provided by the UTM coordinates and compares the location to the work type using the lookup table of the library 165. The document retrieval system 150 checks whether the worker 112 has sufficient permissions to access the documents in the library 165, for example, whether the worker 112 has permission to access the work type nature of the 2D drawing, and / or whether the worker 112 has permission to access only the 2D drawing or the 2D rasterized drawing. If the worker 112 has the appropriate permissions, the document retrieval system 150 retrieves the relevant drawing and sends the document to the mobile device 110 of the worker 112 in a timely manner.

[0111] For example, if during the course of work, the worker 112 encounters one or more problems, the worker 112 can use the document retrieval system 150 to report the problems. The worker 112 obtains and communicates each problem. The worker 112 can propose a solution to the problem and preferably solve the problem on the spot. For example, the worker 112 takes a photo showing the problem, annotates the photo on the mobile device 110, and sends a package including the worker 112's job identification and metadata to the document retrieval system 150.

[0112] The document retrieval system 150 uses the metadata received from the worker 112 to determine which 2D drawings or 2D rasterized drawings are relevant to the problem using the lookup table of the library 170. After the worker 112 controls the system for granting access, the document retrieval system 150 sends the relevant 2D rasterized drawing or 2D drawings back to the mobile device 110. The worker 112 can then annotate the 2D rasterized drawing to send back to the document retrieval system 150 by showing the detailed location of the problem and, if necessary, adding video, images, and / or voice recordings.

[0113] In the case where the mobile device cannot receive GPS signals, errors in the metadata can make it difficult for the document retrieval system 150 to locate the correct 2D drawing. As described above, in this case, a reference device 115 with a unique identifier on site and pre-positioned within the coverage area can provide its known reference position. The mobile position is provided by an offset obtained from the relative position of the mobile device 110 with respect to the reference device 115. Recently, Apple's ultrawideband has tripled the accuracy (from 9 meters to 27 meters) and can be used on watches.

[0114] As another example, the exemplary embodiment may be used to classify a worker's problem and initiate a business process. Problems reported from the field and received by the document retrieval system 150 may be caused by any number of reasons (e.g., incorrect design, unsuitable or defective materials or products, the need to replace a used product or replacement part, broken materials, improper installation, and the need to remove previous work from another job before work can continue, etc.).

[0115] Embodiments may provide links from received example problem reports to related business processes. For example, when a modification is approved as a design change, the document retrieval system 150 may initiate a business process for editing one or more 2D drawings. When an issue with a material or product is approved, the document retrieval system 150 may initiate a business process for supplier and / or receiving issues. When approved as an installation issue or out-of-order work, the document retrieval system 150 may initiate a business process for rework with potential resupply.

[0116] Embodiments may achieve location accuracy of documents in the document retrieval system 150 for a given field. Typically, the location accuracy of field documents is defined by initially assigned geodetic reference points.

[0117] The site plan designer may access the received package with the annotated 2D rasterized drawing. The package contains the location provided by the received metadata and the identifier of the reference device 115, wherein the metadata is provided directly by the mobile device 110 or determined by the offset from the reference device 115 to the mobile device 110. The document retrieval system 150 annotates the location on the 2D rasterized drawing.

[0118] The document retrieval system 150 analyzes the deviations between different metadata locations of the received document packages to understand the cause of the deviations. In general, the resolution and accuracy of the mobile device 110 and / or the reference device 115 play a role. If the difference between the two scales of the device's geographic location is small, the resolution of the mobile device 110 or the reference device 115 is considered to be high. If the mobile device 110 or the reference device 115 produces the same results for the same measurement under similar conditions at different times, the mobile device 110 or the reference device 115 can be considered to have good accuracy. In contrast, if small changes in the received signal produce large differences in the measurement, the mobile device 110 or the reference device 115 can be considered "sensitive".

[0119] Given the accumulated results obtained from the field by each mobile device or reference device, the system 100 can selectively adjust the geographic location of the 2D drawings in the library or metadata from a given device. According to an embodiment, the geographic location of the 2D drawings in the library 165 can be improved over time.

[0120] According to an embodiment, all communications from the mobile device 110 to the 2D drawing library 165 may be saved to ensure traceability and auditability.

[0121] According to an exemplary embodiment, document retrieval system 150 is a retrieval system for 2D drawings. The following detailed description explains the purpose of document retrieval system 150, how errors can be corrected, and how embodiments make the system traceable and auditable.

[0122] Figure 2200 is a flowchart of an exemplary method embodiment for using a document retrieval system. It should be noted that any process description or box in the flowchart should be understood to represent a module, segment, code portion or step including one or more instructions for implementing a specific logical function in the process, and alternative implementations are included within the scope of the present invention, wherein, as can be understood by a person with appropriate skills in the field of the present invention, depending on the functions involved, the functions may not be executed in the order shown or discussed, including being executed substantially simultaneously or in the reverse order.

[0123] As shown in box 210, a library is established for 2D drawings in the document retrieval system. For example, the documents of the library can be updated / modified to refer to predetermined document standards (including geographic location coordinates and measurement units, etc.). As shown in box 220, a mobile device is selected to access the library. For example, the mobile device can be such a mobile device of a staff member, and the staff member uses the mobile device to submit one or more documents to the library and / or access one or more documents from the library at a site that is geographically far away from the library. As shown in box 230, a reference device is selected and set to provide a geographical reference to the mobile device. As shown in box 240, the mobile device accesses the 2D drawing from the library. As shown in box 250, through the mobile device, the staff uses the document retrieval system to communicate a problem or current completion status. For example, the problem can be a difference between the accessed document and the corresponding feature represented by the accessed document on the site, such as a missing feature, or a difference in position, size and / or direction relative to the accessed document. The staff member can indicate the difference through annotations on the submitted and / or accessed documents.

[0124] The document retrieval system 150 for 2D drawings and 2D rasterized drawings may operate with a document check-in / check-out system so that the document retrieval system 150 may be collaborative and provide a current version of the 2D drawing in the library 165 .

[0125] After the existing document repository 165 is ready to operate with the document retrieval system 150 , the document retrieval system 150 may be used with the existing document repository 165 .

[0126] The document retrieval system 150 may employ measurement units of a conventional library. For example, a 2D drawing may be drawn using either the metric system of units or the imperial system of units, and the system of units initially used is maintained by the document retrieval system 150.

[0127] The document retrieval system 150 can operate on specific aspects of design and craft. Table 1 shows an exemplary template associated with metadata identifying the design and craft of a document:

[0128] Table 1: Design and Trade Document Fields

[0129]

[0130] With respect to the Craft field, various 2D drawings convey specific aspects of the design to qualified users by the nature of the craft. For example, an aspect or nature of the craft may be identified by a one or two letter alphabetic symbol. Examples of specific aspects of a design commonly found in 2D drawings include:

[0131] ○ Site Plan - SP: A site plan is an architectural drawing that provides an overview of a property or site. The SP typically shows the location and orientation of buildings, roads, parking areas, landscape features, and other elements of the site.

[0132] ○ Existing Conditions - EX: Existing conditions refer to the current state or characteristics of a site or building before any proposed changes or construction occurs. Existing condition drawings typically depict the layout of existing structures, topography, utilities, vegetation, and other relevant features.

[0133] ○ Layout - L: Layout refers to the arrangement or positioning of various assets within a design. A layout can include many types of entities.

[0134] ○ Demolition-D: Demolition refers to the illustration of areas, asset components, or entire assets that are proposed for removal or demolition.

[0135] ○ Subslab - SS: The subslab generally refers to the area located below the concrete slab of a property. The subslab includes materials and systems installed below the slab, such as insulation, vapor barriers, and utility piping.

[0136] ○ Architectural-A: Architectural drawings show the layout and design of the asset, including walls, floors, and other building elements.

[0137] ○ Life Safety - LS: Life safety refers to the measures and features included in the design to protect occupants from harm during an emergency or hazardous situation. Life safety includes elements such as fire exits, emergency lighting, and other safety features.

[0138] ○ Structure-S: Structural drawings depict the framework and load-bearing components of a structure such as columns, beams, foundations, and structural connections.

[0139] -Structural Steel: Structural steel drawings focus specifically on the fabrication and erection of steel structures, including steel beams, columns, connections, and other steel components.

[0140] ○ Electrical - E: Electrical drawings show the electrical systems within a building, including power distribution, lighting fixtures, sockets, switches, and wiring diagrams.

[0141] ○ Mechanical-H: Mechanical drawings represent mechanical systems and equipment, such as HVAC (heating, ventilation, and air conditioning), piping, ductwork, equipment layout, and ductwork.

[0142] o Piping-P: A piping drawing shows the water supply, drainage, and waste disposal systems within a building, including pipes, fixtures, valves, and other plumbing components.

[0143] ○ Fire Safety-FS: Fire safety drawings detail fire and life safety systems such as fire alarms, fire sprinklers, fire exits, emergency lighting, and evacuation plans.

[0144] - Fire-FP:

[0145] -Fire Alarm-FA:

[0146] ○Kitchen / Catering Services-K:

[0147] ○ Landscape-LA: Landscape drawings depict the design and arrangement of outdoor spaces, including elements such as trees, plantings, walkways, and hardscape features such as walls and fences.

[0148] ○ Civil - C or CE: Civil drawings are related to site development and infrastructure, including site plans, grading, roads, parking lots, utilities, and drainage systems.

[0149] ○ Telecommunications - T: Telecommunications drawings represent the layout and configuration of communications systems, including data networks, telephone lines, fiber optic cables, and equipment placement.

[0150] The specific type of work required for an asset depends on the nature of the design and requirements of the asset being represented. The letters used to represent types of work may vary depending on country and company usage.

[0151] In addition to 2D drawings that include floor plans, the document retrieval system 150 can also manage several other types of 2D drawings used in various disciplines that do not typically include floor plans. Here are some common examples:

[0152] ○ Legend: The legend provides explanations or key information related to the symbols, abbreviations or other elements used in the reference 2D drawing. The legend helps users understand the meaning of various graphical representations or annotations.

[0153] ○ Notes: Notes are text annotations that provide additional information or instructions related to a referenced 2D drawing. Notes can clarify specific details, specify materials, provide dimensions, or convey important information related to other 2D drawings.

[0154] ○ Schedule: A schedule is a tabular representation that organizes and lists information related to various components or elements in a referenced 2D drawing. For example, in an architectural drawing, a schedule may include a list of doors, windows, fixtures, or finishes, and their corresponding quantities, sizes, or specifications.

[0155] ○ Detail: A detail is an enlarged view of a specific part or assembly that references a 2D drawing. A detail provides a more detailed representation of a specific area, highlighting a particular feature, connection, or construction method. Details are often used to convey precise information related to key aspects of a design.

[0156] ○ Elevation: An elevation is a 2D representation of the vertical face or front of a building or structure. Elevations show the exterior of a building from different perspectives, highlighting features such as windows, doors, and architectural details.

[0157] ○ Section: A section is a 2D drawing that cuts through a building or object to show its internal structure or components. Section drawings provide views that help users visualize the relationship between different parts and understand how these parts fit together.

[0158] ○ Diagram: A diagram is a simplified graphical representation used to illustrate a concept, relationship, or process. Diagrams are often used in fields such as engineering, electronics, and system design to explain complex systems or workflows with reference to 2D drawings.

[0159] ○ Isometric Drawing: An isometric drawing is a 2D representation of an object or structure drawn in a 3D perspective. Isometric drawings provide a realistic representation of the spatial layout of an object and are often used for engineering and technical illustrations.

[0160] ○ Rendering: A rendering is a highly detailed and realistic 2D drawing or image that depicts what a design or space will look like once it is completed. Renderings typically incorporate lighting, shading, and textures to provide a visual representation of the final product.

[0161] The types of 2D drawings listed above can be used in conjunction with one or more 2D drawings within a specific type of work, with some exceptions such as initial pages and summaries. These types supplement the information that the user needs to properly understand the task to be performed. 2D drawings are organized into a single 2D drawing or a package of several 2D drawings based on the type of work.

[0162] Figure 4 4 is a flow chart of an exemplary method for adding geographic reference data to a document. As shown in block 410, an existing geographic reference point is located in a current 2D drawing. As shown in block 420, the geographic reference point is converted to UTM coordinates.

[0163] A processing order is defined to add new geographic reference points to the 2D drawing, as shown in block 430. For example, for each drawing, new geographic reference points are added to the 2D drawing, the layout plan is used for horizontal coordinates, columns in the drawing are used as reference points within the asset, and sections and elevations are used to identify defined layers in the asset, starting with the elevation of the lowest layer in the asset.

[0164] As shown in block 440, geographic reference points are added to the corners of the 2D drawing and a true north coordinate system is defined for the 2D drawing. As shown in block 450, polylines of the boundaries of the 2D drawing are extracted. As shown in block 460, a minimum quadrilateral space oriented north-south and east-west and enclosing the four corners of the 2D drawing is defined. For example, in a DWG drawing, the lower left corner of the drawing is the origin (0, 0, 0), and the edges of the drawing constitute the selected polylines. The UTM coordinates of each of the four corners of each 2D drawing are calculated, as shown in block 470.

[0165] For documents that do not already have latitude and longitude, the document retrieval system 150 can add latitude and longitude to some reference points. Reference points with geographic coordinates (geodetic points) are often already available on some 2D maps. Different geographic systems can be used, the most common being the Universal Transverse Mercator (UTM), which represents:

[0166] ○Within the reference median of the 6 degree section of the Earth,

[0167] ○ Latitude and longitude in meters.

[0168] ○ Angles are expressed in degrees, minutes, and seconds, but some expressions may require radians.

[0169] ○Altitude is measured in metres with reference to sea level.

[0170] In order to submit an application for a license on an asset, the documentation should include:

[0171] ○ some geodetic points on the plot or definition points of a unified plot,

[0172] o the segment length and direction of each segment (DMS) along the perimeter of the boundary of the parcel or uniform parcel, and

[0173] o When the segment is not linear, the length of the arc with center and radius.

[0174] ○ Height data may also be obtained through 2D drawings representing elevations and sections submitted as a required part of the permit application for the property.

[0175] Document retrieval system 150 can manage several measurement methods ("measurement units"). Several different unit systems are used by different regions. Document retrieval system 150 references the unit systems most commonly used by surveyors internationally, namely, the Universal Transverse Mercator (UTM) and DMS (degrees, minutes, seconds) systems.

[0176] UTM divides the Earth into 60 zones, each spanning 6 degrees of longitude. Each zone has a unique central meridian and its own set of coordinates. Locations use a combination of the zone number, easting (E), and northing (N). For example:

[0177] -N 123456.7890 refers to the north direction and represents the distance north of the equator in meters, that is, 123456.7890 meters, with an apparent resolution of 4 decimal places.

[0178] -E 8765.4321 refers to the easting and represents the distance east of the central meridian of the UTM zone, which is 8765.4321 meters, with an apparent resolution of 4 decimal places.

[0179] The DMS format (degrees, minutes, seconds) is also used for mapping. The document retrieval system 150 converts the latitude in the DMS format from degrees to meters, where the latitude value is multiplied by the approximate conversion factor of 111111 meters / degree.

[0180] The document retrieval system 150 converts the longitude in the DMS format from degrees to kilometers according to the following formula:

[0181] 1 degree longitude = (π / 180) * radius * cos(latitude) (Equation 1)

[0182] Wherein, radius is the radius of the earth (about 6371 kilometers). DMS is far less accurate than the UTM system and is mainly used for orientation of sectors along the boundaries of unified land masses.

[0183] Permit applications require a professional surveyor to determine:

[0184] ● measuring a number of reference points (geodetic points), roughly the corners of a uniform plot of land, with high resolution (4 decimal places (i.e., tenths of a millimeter)),

[0185] the boundaries of the parcel or parcels of land on which work will be carried out on the property, and

[0186] ● Each segment of the boundary of the uniform plot will be represented according to:

[0187] o If straight, each segment is represented by its length and the angle (in degrees, minutes and seconds) that the segment's line makes with the major axes (N, S, E, W),

[0188] ○ If non-linear, each segment is represented by an arc with its radius, the location of the centre and the length of the arc. The outline of the foundation and / or the top view layout of the finished asset is marked with reference points (geodetic points) at some corners of the asset. Typical resolution is 2 decimal places (cm) in meters. The layout of the asset must show that it complies with local regulations regarding: minimum distances from boundaries, overhangs from boundaries to roofs, equipment, etc.

[0189] A site plan can be submitted by reusing an earlier survey of the lot. There may be some variations in permitting a property in different countries, states, and regions (e.g., counties), but the principles generally remain the same. Site plans, layout plans, sections, and elevations need to be part of the 2D drawings required to submit a valid permit application for a property.

[0190] The document retrieval system 150 adds height coordinates to existing drawings in the library 165 and / or newly received documents from the staff 112. It should be noted that the height of the reference point is not necessarily indicated on the site plan or layout plan. Instead, sections and elevations are typically set to measurements referenced to the ground plane. Each important horizontal plane is measured, such as the foundation, underground floor, ground floor, first floor, and each subsequent floor up to the top of the asset. These measurements use the same system (metric or imperial) as other 2D drawings. For example, in the imperial system, the lowest layer indicated on the 2D drawing refers to sea level in feet and 0 inches. The next higher layer is referenced to sea level in feet and inches. When the ground is tilted, the elevation shows the portion of the asset that can be seen from the perspective, but is typically referenced to the lowest horizontal plane of the asset.

[0191] According to the first embodiment, the document retrieval system 150 determines the processing order to add geographic reference points to the 2D drawing. Generally, each 2D drawing has its own scale. The scale varies in a wide range, for site plans, the scale is about 1 / 1000; for layout plans, the scale is about 1 / 500; for more detailed plans, the scale is about 1 / 100 and 1 / 50. Generally, the 2D drawings provided by the staff to obtain installation details are not geo-referenced because the 2D drawings are used in many different locations.

[0192] 2D drawings use vector graphics. It is possible to zoom in on a portion of a 2D drawing and maintain a very high level of accuracy. The zoom function can typically magnify a 2D drawing by 40,960 times, without being limited by the pixel size. This enables new georeferenced points to be placed on features of the 2D drawing with high accuracy. In contrast, the accuracy of a 2D rasterized drawing is limited by the pixel size used for the raster. For a 2D rasterized drawing, points on a lower scale drawing will typically have better accuracy than points on a higher scale drawing. Preferably, to facilitate the generation of reference points for each 2D drawing, the drawing with the largest scale is converted first, followed by drawings with smaller scales.

[0193] The system uses larger scale reference points, such as those on a layout plan with two decimal places of resolution, to select and reference additional points on the same layout plan, particularly on the perimeter of the asset where piles are typically located for foundations. Depending on the asset layout, the piles are also planned to support specific loads within the interior sub-slabs of the asset. These additional reference points are usually sufficient to step from the layout 2D drawing to the 2D drawings with lower scales and determine the UGM coordinates for each 2D drawing.

[0194] The document retrieval system 150 uses two different methods to reference the 2D drawings to determine the coordinates (true north, south, west, east) of each 2D drawing of the asset. The first method uses a geodetic point and angle, i.e., a geodetic point with latitude and longitude represented in UGM. The first method uses the angle in degrees, minutes, and seconds indicated by a line segment of a boundary represented on the 2D drawing (preferably selected from a large segment). The first method uses the section and elevation of the asset to reference the height separately.

[0195] The second method uses two geodetic points with latitude and longitude coordinates, here (x1, y1) and (x2, y2). The reference level chosen for the corresponding heights z1 and z2 is the same. If necessary, the second method can use isoclines when the values ​​of z1 and z2 are different and not constant.

[0196] The north-south and east-west directions are determined by providing a third point that is the solution to the following equation:

[0197] (x-x1) 2 +(y-y1) 2 =(x2-x1) 2 (Equation 2)

[0198] (x-x2) 2 +(y-y2) 2 =(y2-y1) 2 (Equation 3)

[0199] Geometrically, a point is the intersection of two circles, one centered at (x1, y1) and the second centered at (x2, y2), where (x1, y1) and (x, y) define the horizontal direction parallel to the equator, and (x, y) and (x2, y2) define the vertical direction parallel to the north-south direction. Elevations are referenced using the asset's sections and elevations, respectively.

[0200] The document retrieval system 150 finds the geographic locations of the 4 corners of the 2D drawing by defining the smallest quadrilateral space that encompasses the entire space of the 2D drawing, properly oriented north-south and east-west, and selecting the intersection points between the corners of the 2D drawing and the quadrilateral space. Add new reference points at the corners of the 2D drawing, along the asset, and within the asset if necessary, in a new reference in meters defined by the previous reference points. Create an ordered list of the corners of the 2D drawing, such as in the order bottom left, top left, top right, bottom right.

[0201] The document retrieval system 150 separates the layers of multiple drawings including the common 2D drawings by creating a 2D drawing at each layer. Each of these drawings at each layer maintains the scale and reference point, and maintains the identification of the trades involved at each layer (e.g., HVAC, plumbing, mechanical, security systems, communications, etc.), which is usually used as a standard for each country.

[0202] The document retrieval system 150 creates a lookup table of all 2D drawings of the scene. Figure 3 An exemplary data structure of the lookup table 300 is shown. The top attributes of the cover 305 may include a summary list of sites, assets, and 2D drawings.

[0203] For example, the asset lookup table 300 may have one row for each 2D drawing or 2D drawing package. In this example, the lines indicate: the document title as indicated in the title box, an identifier 324 for each 2D drawing, the plane layer (starting from the lowest layer and going to the highest layer) 310, the layer index 330, the height 340, the nature of the type of work of the 2D drawing (e.g., life safety) 320, the letter symbol (e.g., LS) 322, its number (e.g., 02 for Life Safety LS 02) 324, and the geographic coordinates (latitude, longitude) of the 4 corners of the 2D drawing (lower left 351, upper left 352, upper right 353, and lower right 354).

[0204] The document retrieval system 150 determines whether the provided geographic coordinates are within or outside any 2D mapping coordinates of the asset contained in the library 165. For example, for a horizontal line between the upper left corner (x1, y1) and the upper right corner (x2, y2) of the 2D represented in UGM coordinates, the document retrieval system 150 calculates the slope (m) of the line passing through the points (x1, y1) and (x2, y2) using the following equation:

[0205] m = (y2-y1) / (x2-x1) (Equation 4)

[0206] To determine the point (x n ,y n ) lies on one or the other half-plane defined by the line passing through the points (x1, y1) and (x2, y2). The document retrieval system 150 uses the slope (m) and one of the given points to calculate the equation of the line (in point-slope form):

[0207] (x1, y1): y-y1 = m(x-x1) (Equation 5)

[0208] The document retrieval system 150 calculates the x coordinate (x n ) into the resulting equation to find the corresponding y-coordinate on the line:

[0209] y′ n =m(x n -x1)+y1(Equation 6)

[0210] If y′ n >y n , then the point (x n ,y n ) is above the line, and (x n ,y n ) is not inside the 2D plot. If y′ n <y n , then the point (x n ,y n ) is below the line, and the next boundary (x3, y3) is calculated. Successive evaluations along the 4 boundaries depicted by the corners of the 2D plot will either fail during the evaluation interrupting the process, or succeed and display the provided point (x n ,y n ) is located within a specific 2D drawing.

[0211] The document retrieval system 150 may be configured to create a rasterized version of each 2D drawing (individually or in a package) having the same title, letter symbol, and number.

[0212] Prepare mobile devices to send appropriate metadata to retrieval systems

[0213] The mobile device 110 may be prepared to send metadata formatted for processing by the document retrieval system 150. As described previously, there are different types of systems for measuring location, including survey-grade GPS / GNSS, consumer-grade GPS, Wi-Fi positioning, and cellular network data. Many smartphones use a combination of GPS, Wi-Fi positioning, and cellular network data to provide more accurate and reliable location information. These positioning systems intelligently switch between different positioning technologies based on signal strength, availability, and accuracy. The accuracy of positioning is still quite low.

[0214] For situations where the signal is not obstructed, the mobile device 110 can be configured to verify that for each photo, video, or image captured, the mobile device will create metadata of the coordinates (longitude, latitude, and altitude) of the device at the location where the mobile device captured the photo, video, or image, and possibly the orientation of the camera or imaging system. The mobile device can verify that the metadata is accessible and can be output to the mapping library via digital communication.

[0215] The mobile device 110 can also provide altitude data to the document retrieval system 150. Here, the mobile device 110 uses GPS to collect altitude relative to sea level. For example, on an iPhone, the resolution in meters is set with 2 decimal places (cm). Outdoor measurements can have good accuracy, but may not be very precise. In the case of correct signal reception, altitude can be collected outdoors and indoors. However, in the case of poor signal reception, altitude data may not appear on the mobile device 110.

[0216] In some cases, the mobile device 110 may be temporarily disconnected from the document retrieval system 150. The mobile device 110 may then send the photo and package when reconnected to the document retrieval system 150. For situations where the signal is obstructed, the above-mentioned schemes provide poor results. In such locations, such as in a basement, or a series of walls, or a tunnel, alternative methods may be used, such as using a reference device, as described below.

[0217] Prepare reference equipment to send appropriate metadata to the retrieval system

[0218] The reference device 115 may be prepared to send metadata, which is formatted for processing by the document retrieval system 150. As described above, typically, the reference device may be an indoor positioning system (IPS), typically using radio frequency. BLE, UWB, and LTE are often used for indoor positioning. There are also different methods for using signals exchanged between a mobile device and a reference device, such as angle of arrival, angle of departure, signal strength, time of arrival. In the case of using multiple reference devices 115 or mobile devices 110, additional methods such as triangulation may be used. The device may also use some of these methods in combination.

[0219] The reference device 115 can be set up as follows. A reference point is selected using a 2D drawing, and the reference device 115 is placed at the reference point and a defined height, such as on the floor of an asset on site. The mobile device tracks the (x, y) position of the reference device relative to the mobile device. The photo, video, or image location can be determined by using the offset of the mobile device 110 relative to the reference device 115. For short distances and direct line of sight, this position measurement can generally be considered to be relatively accurate. Several reference devices (each with its own identifier) ​​can be used close to each other to extend coverage and provide triangulation to improve accuracy. Metadata positioning is collected by the mobile device and indirectly by the reference device. The metadata can be sent to the library 165 by digital means, using, for example, Wi-Fi on site, along with other metadata and pictures, videos, images, and annotations. Simple testing can be performed on site using specific mobile devices and reference devices to confirm sufficient accuracy.

[0220] The steps for accessing the metadata of the referenced location vary depending on the specific reference device. For example, with AirTag and iOS devices, you can use the Find My app (pre-installed on iOS devices). The same Apple ID must be used for AirTag and iOS devices. On the AirTag's details page, find the Location section and tap to view the AirTag's latest location. (It should be noted that location resolution continues to improve over time). For example, the new UWB chip U2 increases the detection range from approximately 30 feet to 90 feet (9 meters to 27 meters). It is expected that positioning devices such as AirTags will continue to include these improvements. Locations are represented using the World Geodetic System 1984 (WGS 84). Coordinates can be extracted automatically, for example using Python code to extract coordinates using the installed "pyicloud" library, as shown in Table 2.

[0221] Table 2: Example Python code

[0222] import pyicloud

[0223] #Authenticate with iCloud using your Apple ID and password

[0224]

[0225] The system for performing the functions of the document retrieval system 150 described in detail above may be a computer, an example of which is shown in FIG. Figure 5. The system can be implemented in the cloud, and / or as a distributed system. System 500 includes a processor 502, a storage device 504, a memory 506, an input and output (I / O) device 510 (or a peripheral device), a local bus or a local interface 512, and a software 508 defining the functions mentioned above is stored in the memory 506, and the local bus or the local interface 512 allows communication within the system 500. The local interface 512 can be, for example, but not limited to, one or more buses or other wired or wireless connections, as known in the art. The local interface 512 may have additional elements (e.g., a controller, a buffer (cache), a driver, a repeater, and a receiver) to enable communication, which are omitted for simplicity. In addition, the local interface 512 may include an address, a control, and / or a data connection to enable appropriate communication between the components mentioned above.

[0226] Processor 502 is a hardware device for executing software, particularly software stored in memory 506. Processor 502 can be any custom or commercially available single-core or multi-core processor, a central processing unit (CPU), a secondary processor among several processors associated with the present system 500, a semiconductor-based microprocessor (in the form of a microchip or chipset), a microprocessor, or generally any device for executing software instructions.

[0227] The memory 506 may include any one or a combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.) and non-volatile memory elements (e.g., ROM, hard disk drive, tape, CDROM, etc.). In addition, the memory 506 may include electronic, magnetic, optical and / or other types of storage media. It should be noted that the memory 506 may have a distributed architecture, in which components are arranged remotely from each other but can be accessed by the processor 502. Similarly, the processor 502 may be a single processor or a distributed system of multiple processors, for example, two or more servers.

[0228] According to the present invention, software 508 defines the functions performed by system 500. Software 508 in memory 506 may include one or more separate programs, each of which contains an ordered listing of executable instructions for implementing the logical functions of system 500, as described below. Memory 506 may contain an operating system (O / S) 520. The operating system essentially controls the execution of programs within system 500 and provides scheduling, input-output control, file and data management, memory management, communication control, and related services.

[0229] I / O devices 510 may include input devices such as, but not limited to, keyboards, mice, scanners, microphones, etc. In addition, I / O devices 510 may also include output devices such as, but not limited to, printers, displays, etc. Finally, I / O devices 510 may also include devices that communicate through input and output, such as, but not limited to, modulators / demodulators (modems; used to access another device, system, or network), radio frequency (RF) or other transceivers, telephone interfaces, bridges, routers, or other devices. The system may also use network interface 522 to access external resources, such as, for example, cloud-based servers or storage systems.

[0230] When the system 500 is in operation, the processor 502 is configured to execute software 508 stored within the memory 506, transfer data to and from the memory 506, and generally control the operation of the system 500 according to the software 508, as explained above.

[0231] When the functionality of system 500 is in operation, processor 502 is configured to execute software 508 stored in memory 506, transfer data to and from memory 506, and generally control the operation of system 500 according to software 508. Operating system 520 is read by processor 502, possibly buffered within processor 502, and then executed.

[0232] When the system 500 is implemented in software 508, it should be noted that the instructions for implementing the system 500 may be stored on any computer-readable medium for use by or in combination with any computer-related device, system or method. In some embodiments, such a computer-readable medium may correspond to either or both of the memory 506 and the storage device 504. In the context of this document, a computer-readable medium is an electronic, magnetic, optical or other physical device or apparatus that can contain or store a computer program for use by or in combination with a computer-related device, system or method. The instructions for implementing the system may be embodied in any computer-readable medium for use by or in combination with a processor or other such instruction execution system, device or device. Although the processor 502 has been mentioned by way of example, in some embodiments, such an instruction execution system, device or device may be any computer-based system, a system containing a processor, or other system that can obtain instructions from an instruction execution system, device or device and execute instructions. In the context of this document, a "computer readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with a processor or other such instruction execution system, apparatus, or device.

[0233] Such a computer readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device, apparatus or propagation medium. More specific examples of computer readable media (a non-exhaustive list) may include the following: an electrical connection having one or more conductors (electronic), a portable computer floppy disk (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM, EEPROM or flash memory) (electronic), an optical fiber (optical), and a portable compact disk read-only memory (CDROM) (optical). It should be noted that the computer readable medium may even be paper or another suitable medium on which the program is printed, since the program may be electronically captured, for example, by optical scanning of the paper or other medium, and then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

[0234] In an alternative embodiment where system 500 is implemented in hardware, system 500 may be implemented using any one or a combination of the following technologies, all of which are well known in the art: discrete logic circuits having logic gates for implementing logic functions based on data signals, application specific integrated circuits (ASICs) having appropriately combined logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0235] The following are examples of using the document retrieval system 150. In a first exemplary use case, a worker 112 wants to access 2D drawings related to the trade of his current assignment (e.g., electrical, HVAC, or mechanical, etc.). The worker takes a photo of his location, or even a selfie, and sends the photo to the document retrieval system 150. If the metadata is provided directly by the mobile device 110, the metadata is sent to the document retrieval system 150. If the metadata is provided by using a reference device 115, the reference device identifier is also sent in the package.

[0236] Upon receiving the package, the document retrieval system 150 identifies the approximate location. The document retrieval system 150 verifies the authority of the worker 112 so that the worker 112 can access the relevant 2D drawings and / or 2D rasterized drawings of his work. The document retrieval system 150 compares the height measurement in meters received from the mobile device with the height of the 2D drawing (converted to meters if necessary). The document retrieval system 150 converts the geographic coordinates provided by the metadata of the mobile device 110 from DMS coordinates and height coordinates to UTM coordinates and height coordinates. The document retrieval system 150 compares the converted UTM metadata with the lookup table 300 mentioned above to find out whether the location of the metadata provided by the mobile device is within the four corners of the row of the lookup table 300.

[0237] The document retrieval system 150 retrieves the relevant 2D drawing and the corresponding 2D rasterized drawing when the following conditions are met:

[0238] ●The requested asset is correct

[0239] ●The staff is qualified to perform the required work

[0240] ● Allow workers to access 2D drawings of the type of work to which the asset belongs

[0241] ● Only allow access to 2D graphics in rasterized form or also in native form

[0242] ●Height metadata is located in the horizontal plane of the 2D drawing

[0243] ● The location of the provided metadata is within the corners of the 2D plot

[0244] If the criteria are met, the document retrieval system 150 sends the retrieved document back to the mobile device 110 .

[0245] In a second exemplary use case, a worker 112 encounters field problems with different causes. The worker 112 seeks to collect and communicate each problem, and may propose solutions to solve the problem, preferably solving each problem on site if possible. The worker 112 uses the mobile device 110 to take a photo of each problem, and annotates the photo or image using edits such as arrows, bubbles, magnifying glasses, text, and voice recordings, if necessary. For example, on an Apple mobile device, the worker 112 takes a photo of the problem, moves the photo from the photo application to the desktop of the mobile device, opens the photo using the iOS application "Preview", clicks "Tools" in the toolbar, selects and uses the appropriate tool (arrow, bubble, magnifying glass, text, etc.) to annotate, indicates the type of work of the task he is working on and is experiencing the problem, saves the annotated photo, and sends the photo with his annotations to the document retrieval system 150. Optionally, the worker may also send other data, such as videos and images.

[0246] As mentioned above in Use Case 1, the document retrieval system 150 uses the lookup table 300 of the 2D drawing library 165 with the location metadata received from the worker 112 to determine which 2D drawings are relevant to the type of work performed by the worker 112. The document retrieval system 150 retrieves the relevant 2D drawings as follows.

[0247] Figure 6 The graphs shown compare the maximum geographic area covered by a given paper size format at a given scale with the measured horizontal error. In these calculations, the paper size is larger than the drawing size, but gives good estimates for current mobile devices and reference devices. Studies have shown that in outdoor urban environments, the accuracy of Apple's mobile devices is within 7 to 13 meters, which is for comparison with Garmin's GPS Map 66, which has a root mean square error (RMSE) of less than 1 meter. Most Apple systems use non-military GPS signals. Pre-operational signals are already available on iPhone 14 and other mobile phones, and pre-operational signals are used as dual bands. As satellite constellations increase and become operational, the accuracy of mobile devices (including improvements in altitude accuracy) is expected to improve significantly. As the filing date of this document, the United States, Galileo, BeiDou and QZSS have approximately 61 operational satellites in the total constellation. The planned launch represents an additional 67 satellites, doubling the number of satellites that can be within the line of sight of the mobile device. It was found that the accuracy of the reference device is generally significantly better than the accuracy from the mobile device. In the case where the estimated accuracy of the location is within 94.94 square meters, the chart 600 determines the paper size of the 2D drawing relative to the current size multiple of the accuracy area of ​​the provided metadata. In the case where signal reception is obstructed, finding the error percentage of the correct 2D drawing may require multiple reference devices.

[0248] The mobile device 110 of the worker 112 who initiated the request receives the relevant 2D drawing or 2D rasterized drawing which is sent back to the worker 112. Documents are usually sent by the document retrieval system 150 as 2D rasterized drawings to facilitate easy annotation and speed up delivery. The worker 112 annotates the 2D drawing by showing the detailed location of the problem. The annotated 2D drawing is added to the package of photos, videos, images, metadata, annotations and sent back to the system.

[0249] Issues received by the document retrieval system 150 may cover many reasons, such as incorrect design, unsuitable or defective materials or products, the need to replace used products or replacement parts, broken materials, improper installation, and the need to remove previous work from another job before work can continue.

[0250] After the issue and suggestion are communicated, upon approval, links to relevant business processes are provided. For example, if the designer team approves the issue as a design change, a link initiates the business process for editing the 2D drawing. If the issue is approved as a material or product issue, a link initiates the business process for supplier and / or receiving issues. If the issue is approved as an installation issue or out-of-sequence work, a link initiates the business process for rework with potential resupply.

[0251] The initial accuracy of the positioning is defined by a 2D drawing with precise locations of geodetic points and added reference points. The designer receives the annotated drawing and its metadata from the worker and places it on the 2D drawing. The locations provided by the received metadata and the locations annotated on the 2D drawing are stored, and the deviations are analyzed to understand their root causes, for example, whether the geographic location of the drawing in the 2D drawing library needs to be adjusted, whether the geographic location received from the mobile device's own metadata needs to be compensated in the future, or whether the geographic location received from the indirect metadata from the reference device to the mobile device needs to be compensated in the future. This is a cumulative process of multiple issues received from the field that will improve the geographic location of different parts of the asset in the library. Although the above examples describe the actions of a designer, some or all of these steps can be performed automatically by the document retrieval system 150.

[0252] The document management system may save all communications from the mobile device 110 to the 2D drawing library 165 for traceability and auditability. Similarly, all states of the 2D drawings in the library 165 may be traceable and auditable.

[0253] The present embodiment provides many advantages over previous document retrieval systems. Workers using 2D drawings in the field represent the majority of team members in many fields of business, such as architecture, engineering and construction (about 85%), cities, regions and public services (about 50%), fixed assets (about 50%) and manufacturing (about 10% or more), etc.

[0254] An individual may use an embodiment, for example, for building audits, to perform energy performance estimates to assess whether an asset needs to be refurbished, for day-to-day duties at a plant, and to determine the safety and status of equipment on large assets (leaks, noisy equipment, heat or smoke). Embodiments are similarly suitable for collaborative use with others located on-site or remotely located.

[0255] For all of its underlying complex functionality, embodiments are relatively simple to deploy, involving familiar mobile devices (tablets, phones), and relatively few user clicks to communicate issues between workers and field supervisors / managers or off-site resources (e.g., design offices, planning offices, purchasing offices, etc.). Embodiments can leverage existing annotation software to create metadata that is typically pre-loaded on mobile devices.

[0256] Advantageously, embodiments provide rapid accessibility to relevant 2D drawings and 2D rasterized drawings, i.e., instant control of access permissions granted to staff members is provided through the system, as well as instant communication with a data package containing all the factual resources to resolve the issue. No longer does it require someone to be in the office to handle document requests, or to search through all 2D drawings and 2D rasterized drawings on a helpline in order to provide the required document.

[0257] Thanks to the check-in / check-out method, each 2D drawing of the library is unique. Designers can add all annotation data received from workers on 2D rasterized drawings as layers to the correct 2D drawing, since the representation of these annotation data is strictly the same. No special conversion is required for annotation layers.

[0258] The accuracy of the library improves with use: the more questions the library receives from the field, the more accurate the deviation analysis will be using the geolocation of a 2D drawing of that field.

[0259] Embodiments can resolve multiple issues with different workers on the same 2D drawing. When multiple workers encounter different issues related to the same 2D drawing, the 2D rasterized drawing is copied from the same original 2D drawing. This enables each of these workers to annotate his copy of the 2D rasterized drawing. Each set of annotations provided by a worker to the designer can be represented by a layer and added to the 2D drawing until a design change is made. This also ensures that the uniqueness of the original 2D drawing and subsequent edits is maintained through a check-in check-out method.

[0260] Embodiments improve the agility and efficiency of business processes. As each problem collected by a worker is classified, an embodiment can initiate a related business process. The recipient who handles the subsequent steps of the business process can access all relevant and associated data to implement the solution. This agility speeds up problem solving and improves productivity.

[0261] It is obvious to those skilled in the art that various modifications and changes may be made to the structure of the present invention without departing from the scope or spirit of the present invention. In view of the foregoing, it is intended that the present invention encompass modifications and changes of the present invention as long as these modifications and changes fall within the scope of the appended claims and their equivalents.

Claims

1. A method for accessing a two-dimensional (2D) drawing in a document retrieval system from a work site, comprising the following steps: configuring a reference device at the work site to provide geo-reference to a mobile device at a remote work site; receiving, by a document retrieval system in communication with a document repository, the annotated document from the mobile device, the document repository being configured to store a plurality of 2D drawings; accessing, by the document retrieval system, a 2D drawing in the document repository based on the annotated document; and providing a rasterized version of the 2D drawing to the mobile device, Wherein, the work site is located away from the document retrieval system.

2. The method according to claim 1, further comprising the steps of: The document retrieval system receives a current task completion status from the mobile device.

3. The method according to claim 1, wherein: The first group of planning, management, renovation, auditing, construction, maintenance and demolition projects from the second group of architecture, engineering and construction are located at the said work site.

4. The method according to claim 1, wherein: The annotated document includes annotation metadata including location data received from the reference device.

5. The method according to claim 1, wherein: The annotated document includes a photograph.

6. The method according to claim 4, further comprising the steps of: The 2D drawing is annotated according to the annotated document.

7. The method according to claim 6, further comprising the steps of: By means of the document retrieval system, the annotated 2D drawings and / or the annotated submitted drawings are stored.

8. The method according to claim 1, wherein: The annotated document indicates differences between the accessed 2D drawing and corresponding features represented by the accessed 2D drawing at the work site.

9. The method according to claim 1, wherein: The 2D drawing includes metadata identifying the design and type of work.

10. The method according to claim 9, wherein: The metadata identifying the design and the type of work includes a field selected from the group consisting of: a type of work field, a form number, a form name, and a computer-aided drawing (CAD) stage.

11. The method according to claim 1, wherein: The 2D drawing is a vector drawing.

12. The method according to claim 1, wherein: The reference device includes an indoor positioning system IPS device.

13. A method for adding geo-referenced data to a two-dimensional (2D) drawing, comprising the steps of: Finding existing georeferenced points in the 2D drawing; Converting the geographic reference point into Universal Transverse Mercator (UTM) coordinates; and Adds a new georeferenced point to the 2D drawing.

14. The method according to claim 13, further comprising the steps of: Adds geographic reference points to the corners of the 2D drawing.

15. The method according to claim 13, further comprising the steps of: defining a true north coordinate system for the 2D drawing; Extracting a polyline of a boundary of the 2D drawing; A minimum quadrilateral space defining the 2D drawing oriented north-south and east-west; as well as Calculate the UTM coordinates for each corner of the 2D drawing.

16. The method according to claim 13, wherein: The 2D drawing is a vector drawing.

17. A system for accessing a remotely located two-dimensional (2D) drawing via a mobile device at a work site, comprising: a reference device configured to provide a geographic reference to the mobile device; a document library configured to store a plurality of 2D drawings; a document retrieval system configured to access a plurality of 2D drawings in the document repository, wherein the mobile device is configured to provide the annotated document to the document retrieval system, the document retrieval system is configured to access a 2D drawing in the document repository based on the annotated document, and the document retrieval system is configured to provide a rasterized version of the 2D drawing to the mobile device.

18. The system of claim 17, wherein: The annotated document includes annotation metadata including location data received from the reference device.

19. The system of claim 17, wherein: The annotated document includes a photograph.

20. The system of claim 17, wherein: The 2D drawing includes metadata identifying the design and type of work.

21. The system of claim 17, wherein: The metadata identifying the design and the type of work includes a field selected from the group consisting of: a type of work field, a form number, a form name, and a computer-aided drawing (CAD) stage.

22. The system of claim 17, wherein: The 2D drawing is a vector drawing.

23. The system of claim 17, wherein: The reference device includes an indoor positioning system IPS device.

Images