Computer system, method and medium for property inspection

Abstract

A computer system, method, and medium for property inspection. Digital media is received from one or more user devices via a network in a computer vision image analysis system and a determination is made for an environment type associated with the received digital media. One or more objects located in the determined environment and present in the received digital media are determined. And determining whether an object exists in the received digital media according to the determined environment type based on the rule group. Certain implementations of the disclosed technology may include systems and methods that train and prompt a user to tell loss details while capturing an audiovisual record of an impairment or loss. Some implementations may also utilize artificial intelligence analysis of the captured audiovisual data.

Description

TECHNICAL FIELD

[0001] The disclosed embodiments relate generally to systems and methods for capturing first notice of loss (FNOL) reports, and more particularly to training and prompting a user to narrate loss details while capturing audiovisual records of damage or loss. Certain implementations of the disclosed technology can include generating a conversational flow to assist a user, and / or utilizing artificial intelligence analysis of captured audiovisual data. BACKGROUND

[0002] When an insurance provider offers insurance to a home or business, they assume the risk that any damage or liability associated with the property will be offset by the premiums paid by the property owner. To strike a good balance between offering competitive prices and managing risk, insurance providers can wish to assess the relative risk of each potential insurable property and / or receive and evaluate documentation of actual claim losses.

[0003] Property inspections and / or loss reports have historically required a trained professional to personally travel to the property for a comprehensive property assessment while recording important details in a report or series of reports. This process has proven to be inefficient and requires training of the professional, as well as travel time and expense for transportation and inspection labor. In some cases, property inspections are not conducted at all (e.g., without seeing the physical property), thus exposing insurance providers and other parties to an unnecessary level of risk.

[0004] With the proliferation of mobile computing devices such as tablets and smartphones, end customers are now able to capture and document property damage themselves without relying on a professional inspector. However, it has proven to be a strange and unintuitive process for users to narrate while filming to report a loss, etc., especially when they are using a rear-facing camera. In a test group, users did not do this even when explicit instructions were provided to explain the video, if they could not see themselves on the screen.

[0005] There is a need to train and prompt a user to narrate loss details while capturing audiovisual records of damage or loss in order to capture the appropriate information. SUMMARY

[0006] The objects and advantages of the illustrated embodiments will be set forth in or made apparent from the description that follows, and will be realized and attained by means of the apparatus, systems and methods particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0007] According to certain implementations of the disclosed technology, computer-implemented methods, systems, and non-transitory computer-readable media are provided for initiating and capturing an audiovisual document of an environment. The method, system, and / or computer-readable media are configured to receive, at a mobile computing device, an input command to initiate capture of an audiovisual document of an environment; output, from the mobile computing device: an instruction for a user to speak one or more test phrases; and a user training progress indicator configured to advance in response to an auditory detection of the one or more test phrases. During a training phase, the method, system, and / or computer-readable media are configured to receive an auditory input corresponding to the instruction; and advance the user training progress indicator in response to the received auditory input; and switch to an audiovisual capture phase in response to receiving a predetermined threshold amount of auditory input. During the audiovisual capture phase, the method, system, and / or computer-readable media are configured to capture, by the mobile computing device, an audiovisual document comprising video of the environment and audio of the user narrating.

[0008] Illustrative embodiments relate to an insurance provider preferably receiving information about an insured property from a user of a smart device located at a property site, where the information indicates a risk associated with the property. Based on the received information, the insurance provider determines a risk-adjusted insurance premium for the property to adjust for the indicated risk. In particular, the illustrated embodiments provide an Artificial Intelligence (AI) assistant for the underwriting process. In particular, the AI assistant preferably guides a user through an underwriting and inspection process through a conversational flow so that any user of a smart device with a camera can capture property information to be used in the underwriting process without resorting to expensive, trained professionals. As a result, insurance providers and other parties that benefit from such property data are able to collect important property information faster and more economically than previously possible by trained professionals to determine property values and risk exposures. BRIEF DESCRIPTION OF DRAWINGS

[0009] The appended appendices and / or drawings illustrate various non-limiting, exemplary, inventive aspects in accordance with the present disclosure:

[0010] Figure 1 An example system-level diagram of a communication network is shown that is used with the illustrated embodiments;

[0011] Figure 2 An example system-level diagram of a network device / node that contains at least a portion of the illustrated embodiments and can be used in a communication network such as Figure 1 is shown;

[0012] Figure 3 An example system-level diagram of the illustrated embodiments for performing a property inspection is depicted;

[0013] Figure 4 It shows Figure 3 A flowchart of the operation of the illustrated embodiment;

[0014] Figure 5A It shows that according to Figure 4 A first exemplary screenshot of a user device used to perform a property inspection operation according to the embodiment shown;

[0015] Figure 5B It shows that according to Figure 4 A second exemplary screenshot of a user device for performing a property inspection operation according to the illustrated embodiment;

[0016] Figure 5C It shows that according to Figure 4 A third exemplary screenshot of a user device for performing a property inspection operation according to the illustrated embodiment; and

[0017] Figure 6 It is a flowchart of a method based on some exemplary implementations of the disclosed technology. Detailed Implementation

[0018] The illustrated embodiments will now be described more fully with reference to the accompanying drawings, wherein like reference numerals denote similar structural / functional features. The illustrated embodiments are not intended to be limited in any way to what is shown, as the illustrated embodiments described below are merely exemplary and, as will be understood by those skilled in the art, can be embodied in various forms. Therefore, it should be understood that any structural and functional details disclosed herein should not be construed as limiting, but rather as the basis for the claims and as an indication to teach those skilled in the art to use the discussed embodiments in various ways. Furthermore, the terminology and phrases used herein are not intended to be limiting, but rather to provide an understandable description of the illustrated embodiments.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Although any methods and materials similar to or equivalent to those described herein may be used in the practice or testing of the illustrated embodiments, exemplary methods and materials are described hereafter.

[0020] It must be noted that, unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the” used herein and in the appended claims include plural references. Thus, for example, reference to “a stimulus” includes multiple such stimuli, reference to “the signal” includes reference to one or more signals and their equivalents known to those skilled in the art, and so on.

[0021] It should be understood that the embodiments discussed below are preferably software algorithms, programs, or code residing on a computer-usable medium having control logic, which can be executed on a machine having a computer processor. This machine typically includes memory configured to provide output from the execution of the computer algorithm or program.

[0022] As used herein, the term "software" is a synonym for any code or program that may be present in the processor of a computer host, whether that implementation is in hardware, firmware, or as a software computer product available as a disk, storage device, or downloadable from a remote machine. The embodiments described herein include software for implementing the equations, relations, and algorithms described above. Those skilled in the art will understand further features and advantages of the embodiments shown above. Therefore, unless indicated by the appended claims, the illustrated embodiments are not limited to what is specifically shown and described.

[0023] Some implementations of the disclosed technology can be used to prompt users to narrate a scene while correctly recording damage or loss using the rear camera of their mobile computing device. Experiments on user groups have shown that users are generally not accustomed to narrating during video capture if they cannot see themselves on the screen, even with clear instructions or prompts such as real-time microphone volume displays. A working theory consistent with the experiments is that users perceive themselves as recording "what's on the screen," and if they cannot see themselves in the captured video, they may assume their voice cannot be recorded and therefore there is no need to speak.

[0024] Some implementations of the disclosed technology provide systems and methods for prompting users to speak while capturing video by inserting a user training progress indicator into the video. In some implementations, the user training progress indicator can be configured to advance (e.g., fill bars, circles, or other indicators) in response to the auditory detection of one or more phrases. For example, during the training phase, the mobile computing device can receive auditory input corresponding to instructions to speak one or more test phrases. The user training progress indicator can advance in response to the received auditory input. In response to receiving a predetermined threshold amount of auditory input, the mobile computing device can switch to the audiovisual capture phase to record the loss. Experiments show that users respond very consistently. For example, users initially appear confused, but after reading the instructions, they speak the test phrases, which in turn improves the progress indicator. Once users see the connection between the progress indicator and their speech on the screen, they quickly continue speaking to advance the progress indicator and continue speaking during the audiovisual capture phase to record the loss. After adding the step of the user training progress indicator to the recording process, 100% of the users in the test group correctly described the details of the loss while recording the loss video with the rear camera of the mobile computing device.

[0025] Now we turn descriptively to the accompanying drawings, where similar reference numerals denote similar elements throughout multiple views. Figure 1 An exemplary communication network 100 in which the embodiments shown below can be implemented is depicted.

[0026] It should be understood that a communication network 100 is a geographically distributed collection of nodes interconnected by communication links and segments, used to transmit data between terminal nodes (such as personal computers, workstations, smartphones, tablets, televisions, sensors, and / or other devices, such as automobiles). Many types of networks are available, ranging from local area networks (LANs) to wide area networks (WANs). LANs typically connect nodes via dedicated private communication links located in the same physical location (such as buildings or campuses). WANs, on the other hand, typically connect geographically dispersed nodes via long-distance communication links (such as public carrier telephone lines, optical optical paths, synchronous optical networks (SONETs), synchronous digital hierarchy (SDH) links, or powerline communications (PLCs)).

[0027] Figure 1 This is a schematic block diagram of an example communication network 100, exemplarily including nodes / devices 101-108 interconnected via various communication methods (e.g., sensor 102, client computing device 103, smartphone device 105, web server 106, router 107, switch 108, etc.). For example, link 109 may be a wired link or may include a wireless communication medium, wherein some nodes communicate with other nodes based on factors such as distance, signal strength, current operating state, location, etc. Furthermore, as those skilled in the art will understand, where appropriate, each device may communicate data packets (or frames) 142 with other devices using predefined network communication protocols (e.g., various wired and wireless protocols, etc.). In this context, the protocol consists of a set of rules defining how nodes interact with each other. Those skilled in the art will understand that any number of nodes, devices, links, etc., can be used in a computer network, and the views shown herein are for simplicity. Furthermore, while embodiments are illustrated herein with reference to a general network cloud, the description herein is not limited thereto and can be applied to hardwired networks.

[0028] Those skilled in the art will understand that aspects of the present invention can be embodied as systems, methods, or computer program products. Therefore, aspects of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, which are generally referred to herein collectively as “circuit,” “module,” or “system.” Furthermore, aspects of the present invention can take the form of computer program products embodied in one or more computer-readable media containing computer-readable program code.

[0029] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media will include the following: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, compact disc ROM (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store programs used by or in conjunction with an instruction execution system, apparatus, or device.

[0030] Computer-readable signal media may include propagated data signals containing computer-readable program code, for example, in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and may communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0031] The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, radio frequency (RF), or any suitable combination thereof.

[0032] The computer program code used to perform the operations of various aspects of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, as a standalone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet through an Internet service provider).

[0033] The following describes aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executable via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in the flowchart and / or block diagram blocks.

[0034] These computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing apparatus or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing including instructions that implement the functions / actions specified in the flowcharts and / or block diagrams.

[0035] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other equipment to cause a series of operational steps to be performed on the computer, other programmable apparatus or other equipment, thereby producing a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide a process for implementing the functions / actions specified in the flowchart and / or block diagram blocks.

[0036] Figure 2 This is a schematic block diagram of an example network computing device 200 (e.g., client computing device 103, server 106, etc.), which can be used (or components thereof) with one or more embodiments described herein, for example, as one of the nodes shown in network 100. As mentioned above, in different embodiments, these various devices are configured to communicate with each other in any suitable manner, for example, through communication network 100.

[0037] Device 200 is intended to represent any type of computer system capable of performing the teachings of the various embodiments of the present invention. Device 200 is merely one example of a suitable system and is not intended to impose any limitation on the scope or functionality of the embodiments of the invention described herein. In any case, computing device 200 is capable of implementing and / or performing any of the functions described herein.

[0038] Computing device 200 can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of known computing systems, environments, and / or configurations suitable for computing device 200 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or notebook computer devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers (PCs), minicomputer systems, and distributed data processing environments that include any of the above systems or devices.

[0039] The computing device 200 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computing device 200 can be implemented in a distributed data processing environment, where tasks are executed by remote processing devices linked via a communication network. In a distributed data processing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0040] Equipment 200 Figure 2 It is shown in the form of a general-purpose computing device.

[0041] The components of device 200 may include, but are not limited to, one or more processors or processing units 216, system memory 228, and bus 218 that couples various system components, including system memory 228, to processor 216.

[0042] Bus 218 represents any or more of several types of bus architectures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. By way of example and not limitation, such architectures include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses. Computing device 200 typically includes various computer system-readable media. Such media can be any available media accessible to device 200, and it includes volatile and non-volatile media, removable and non-removable media.

[0043] System memory 228 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 230 and / or cache memory 232. Computing device 200 may also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 234 may be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown, commonly referred to as a "hard disk drive"). Although not shown, disk drives may be provided for reading from and writing to removable, non-volatile disks (e.g., "floppy disks"), and optical disc drives may be provided for reading from or writing data to removable, non-volatile optical discs such as CD-ROMs, digital video disc ROMs (DVD-ROMs), or other optical media. In this case, each may be connected to bus 218 via one or more data media interfaces. As will be further described below, memory 228 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the present invention.

[0044] A program / utility 240 having a set (at least one) of program modules 215 (such as an underwriting module, etc.) can be stored in memory 228, such as, but not limited to, an operating system, one or more applications, other program modules, and program data. Each of the operating system, one or more applications, other program modules, and program data, or some combination thereof, can include an implementation of a network environment. Program modules 215 typically perform the functions and / or methods of the embodiments of the invention described herein.

[0045] Device 200 can also communicate with one or more external devices 214, such as keyboards, positioning devices, displays 224, etc.; one or more devices that enable a user to interact with computing device 200; and / or any device that enables computing device 200 to communicate with one or more other computing devices (e.g., network interface cards, modems, etc.). This communication can be performed via input / output (I / O) interface 222. Furthermore, device 200 can communicate with one or more networks, such as local area networks (LANs), general-purpose wide area networks (WANs), and / or public networks (e.g., the Internet), via network adapter 220. As shown, network adapter 220 communicates with other components of computing device 200 via bus 218. It should be understood that, although not shown, other hardware and / or software components can be used in conjunction with device 200. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) systems, tape drives, and data archiving storage systems.

[0046] Figure 1 and Figure 2 A brief, general description is intended to provide an illustrative and / or suitable exemplary environment in which embodiments of the invention described below can be implemented. Figure 1 and Figure 2 These are examples of suitable environments and are not intended to imply any limitation on the structure, scope of use, or functionality of embodiments of the invention. A particular environment should not be construed as having any dependency or requirement on any one or combination of components shown in the exemplary operating environment. For example, in some cases, one or more elements of the environment may be considered unnecessary and omitted. In other cases, one or more additional elements may be considered necessary and added.

[0047] The exemplary communication network 100 has been shown and discussed above. Figure 1 ) and computing devices 200 ( Figure 2 In the case of [missing information], a description of certain illustrated embodiments of the invention will now be provided. Reference is now made to [missing information]. Figure 3 andFigure 4 Typically, an artificial intelligence (AI)-assisted property inspection system 300 is described, which provides technological improvements to existing underwriting and inspection systems by offering an AI-powered intelligent computer system specially configured to guide end-users through inspections and an underwriting workflow tailored to each user based on their property. The components and processes described below include users downloading a mobile application on their smart devices 101, 105 (e.g., smartphones (iOS or Android), tablets, or other portable computing devices preferably with a display and camera). The property inspection system 300 provides a conversational assistant and user experience, which preferably guides user 105 through a workflow (user conversation flow) in a step-by-step manner, preferably via chat or voice, providing specially tailored instructions to capture photos, videos, and other information requested by the user. The captured photos and videos are sent from user device 105 to system 300, which then sends them to a computer vision application program interface (API) with a computer vision image analysis system 310 for processing. As will be discussed, the computer vision API is configured to identify (extract data) (preferably using optical recognition technology) and record objects, materials, structures, conditions, etc., associated with photos and / or videos received from user device 105. The data extracted by the computer vision API 310 is then sent to a coupled conversational AI / bot service API with a conversational feature extractor system 320 configured to format the user conversation stream based on content identified (and not identified) by the computer vision API 310. Furthermore, it should be understood that during user-server interactions, data captured by the computer vision server 106 from user device 105 is preferably stored in a recording database 330, preferably as a digital record / report.

[0048] The following discussion concerns the technical improvements to existing computer systems, particularly (but not limited to) the analysis and processing of data extracted from user devices 105 in the computer vision API 310 and conversational AI / Bot service system 320 during the underwriting process, in order to preferably identify and generate reports rich in internal and external datasets to add pricing / value data and risk exposure estimates.

[0049] It should be understood that System 300 is described herein to illustrate how it can be used with insurance underwriting tasks; however, System 300 should not be construed as being limited to use with insurance underwriting, as it can be used in any applicable application / use environment. For example, one such use includes the mobile industry, where System 300 is configured to identify and catalog a set of content (e.g., home, office, etc.), which can then be used to provide detailed reports on the content to be moved and its current value (as described below).

[0050] Now for reference Figure 3 The diagram illustrates a simplified overall view of a preferred embodiment, depicting user equipment 105 coupled to server system 300 via one or more networks 100. Server system 300 preferably includes a computer system API 310, a session AI / Bot service system 320, and a record database 300, all preferably interconnected for bidirectional communication with each other. It should be understood that each of the aforementioned components should be understood to include... Figure 2 One or more components of the computer system 200 shown.

[0051] As described above, user equipment 105 should be understood to include a portable computer device, which preferably has network connectivity components (e.g., a cellular transceiver), a display (e.g., a touchscreen display), and a camera configured to capture photos and videos. According to a preferred illustrated embodiment of system 300, user equipment 105 should be understood as a smartphone device or a tablet device.

[0052] The computer system AIAPI 310 is preferably configured to interact with user device 105 to receive captured media (e.g., photos and / or videos) from user device 105 for analysis. In the illustrated preferred embodiment, AIAPI 310 is configured to perform an insurance check identification task (as described herein) on the received media, but should not be construed as being limited to performing an insurance check identification task. In a preferred embodiment, AIAPI 310 is configured to preferably use AI to detect the environment associated with the received media (e.g., kitchen, living room, bedroom, garage, exterior structure, roof, etc.), and more specifically, to detect objects located in that environment (e.g., fireplace, refrigerator, fireplace, lighting components, curtains, exterior structural materials), the position of structures relative to nearby environmental elements (e.g., standing or still water, bushes, landscape grade), recreational objects (e.g., swimming pool, trampoline, etc.). Preferably, AIAPI 310 is further configured to determine that an object does not exist in a particular environment. For example, if the environment is a kitchen, AI API 310 can determine that there is no fire extinguisher; if the environment is a bedroom, AI API 310 can determine that there is no fire / smoke / carbon dioxide (CO2) detector; or if the environment is a swimming pool, AI API 310 can determine that there is a fence and / or some safety device (e.g., a life jacket).

[0053] The AI ​​API 310 is also configured to interact with the conversational AI / Bot service system 320 to essentially instruct the AI ​​API 310 on what it detected and what it did not detect when analyzing media received from the user device 310. This information / data enables the conversational AI / Bot service system 320 to preferably use a pre-configured set of rules to determine the conversational flow of requested follow-up information to be presented to the user device 105. For example, if the detection environment of the received media is a kitchen, and the AI ​​API 310 is unable to detect the brand / model of certain detected kitchen appliances (e.g., stove and refrigerator) and the absence of certain objects (e.g., fire extinguisher, smoke / heat / CO2 detector), then the AI ​​API 310 instructs the conversational AI / Bot service system 320 that this additional information is needed. For illustrative purposes, another example of the capabilities of the AI ​​API 310 includes, if the detected environment (e.g., living room, bedroom, etc.) includes a fireplace, then in addition to detecting objects (e.g., furniture, curtains, etc.) located near the fireplace that may have flammable properties, it can also determine that a protective fireplace door / screen is absent. In this scenario, the Session AI / Bot Service System 320, using its rule groups, will orchestrate the session stream to the user device 105, requesting the type of fire-fighting equipment (if any) installed on the fireplace, and the type of materials used in objects detected near the fireplace.

[0054] Therefore, it should be understood that the conversational AI / Bot service system 320 is configured, preferably using rule groups, to orchestrate the conversational stream of user device 105 using the information provided by AIAPI 310. It should be understood that the conversational stream may include chat formats (including conversation bubbles), Short Message Service (SMS), Mobile Station Module (MSM), email, messaging, and auditory and / or video communication types with user device 105. Examples are provided below. To further understand, AIAPI 310 is also configured to instruct the conversational AI / Bot service system 320 to determine and modify the user interaction experience / conversation on user device 105 to adapt it based on what the camera sees and the data provided by user device 105.

[0055] The record database 330 preferably receives and stores information determined from the AIAPI 310 and the session AI / Bot service system 320. In the context of the insurance industry, this stored information can be used for underwriting purposes (e.g., risk and premium determination, premium renewal, claims determination and adjustment, and other tasks related to insurance underwriting). The record database 330 can also be configured to generate reports on properties to be insured.

[0056] Using some components of the embodiments shown above, now refer to Figure 4 And Figure 5 (and continue to refer to) Figure 3 The operating method will now be discussed. Starting at step 410, the user of user device 105 installs an application (app) orchestrated for their smartphone device 105 to enable the smartphone device 105 to interact with system 300 as described herein. When the user wishes to perform a task using system 300, such as an insurance underwriting task, the user opens the application on their device 105 to initiate the underwriting task (e.g., a homeowner's policy request), step 420. This, in turn, causes the application to interact with system 300 via network 100, such that system 300 and the application provide the user with instructions to begin initiating the requested task, thereby the application preferably guiding the user through a property check in a conversational manner, step 430. Figure 5A The application can activate the camera on user device 105, enabling it to capture video and / or photos of the requested environment (e.g., a kitchen). Figure 5B The media is then preferably transmitted to system 300 for analysis by AIAPI 310 (step 440). It should be understood that, according to the preferred embodiment shown, media capture and transmission can occur simultaneously (in real time).

[0057] Once the captured media is transmitted by user device 105 (step 440) and received by AI API 310 in system 300, it is preferably analyzed by AI API 310 using artificial intelligence technology (step 450) to determine objects (including object materials and conditions) and the absence of objects in the subject environment (e.g., a kitchen), as described above. Also as described above, AI API 310 provides this information to the conversational AI / Bot service system 320 (step 460). As described above, preferably using pre-configured rules, the conversational AI / Bot service system 320 provides the user device 105 ( Figure 5C The session is formatted, and user equipment 105 is requested to provide additional information regarding the initiated insurance underwriting task, step 470. In response to the presented session data from the session AI / Bot service system 320, the data is then preferably sent back from user equipment 105 to AIAPI 310 for parsing and analysis (step 480). The above process preferably continues until system 300 determines that no further relevant data is being obtained from user equipment 105.

[0058] It should be understood that the above process can be performed in real time, whereby the user of user device 105 can capture video simultaneously analyzed by system 300. For example, when the user is capturing video of the kitchen, a conversation bubble (a “conversation stream” sent by the conversation AI / Bot service system 320) will appear on the user’s device 105, requesting certain information (e.g., the brand / model of appliances, requesting capture of fire extinguishers and / or other safety equipment).

[0059] The data captured during the aforementioned auxiliary and adaptive workflow is preferably stored in database 330, step 490. It should be understood that this stored data can be compiled into a comprehensive report, where information captured and identified by AIAPI 310, along with user input (which may include third-party data), is used to enrich the value estimates and risk predictions of the stored data. Examples of such reports include: property address; report date; property contents; condition; materials; and risk items. The report is enriched using external and internal datasets, thereby adding estimates of value and risk exposure to the report.

[0060] Figure 6This is a flowchart of method 600 according to certain exemplary implementations of the disclosed technology. In block 602, method 600 includes receiving an input command at a mobile computing device to initiate the capture of an audiovisual document of the environment. In block 604, method 600 includes outputting an instruction from the mobile computing device for a user to speak one or more test phrases, and a user training progress indicator configured to advance in response to auditory detection of one or more test phrases. In block 606, when in the training phase, method 600 includes receiving auditory input corresponding to the instruction and advancing the user training progress indicator in response to the received auditory input. In block 608, in response to receiving a predetermined threshold amount of auditory input, method 600 includes switching to the audiovisual capture phase. In block 610, and in the audiovisual capture phase, method 600 includes capturing an audiovisual document by the mobile computing device, the audiovisual document including video of the environment and audio of the user's speech.

[0061] Some implementations of the disclosed technology may include transmitting audiovisual documents to a remote server.

[0062] In some implementations, the video portion of an audiovisual document can be captured, at least partially, by the rear camera of a mobile computing device.

[0063] In some implementations, the user training progress indicator can be configured to provide visual or auditory feedback to prompt the user to speak through the audiovisual document.

[0064] According to certain exemplary implementations of the disclosed technology, the audiovisual document may include a representation of structural damage in the environment. In some implementations, the audiovisual document may include a representation of vehicle damage.

[0065] According to certain exemplary implementations of the disclosed technology, a user training progress indicator may include one or more auditory and visual information.

[0066] Some implementations of the disclosed technology may include detecting one or more objects in an environment based on a set of rules. In some implementations, this set of rules may lead to analysis to determine whether one or more objects match one or more pre-defined objects present in the environment. In some implementations, one or more rules may be used to evaluate whether one or more pre-defined specifications can be determined through analysis of one or more objects.

[0067] Some implementations of the disclosed technology may include user instructions for outputting audiovisual documents for capturing the environment from a mobile computing device.

[0068] It should be understood that the systems and methods disclosed herein can provide technical and functional improvements to existing computer systems, including but not limited to providing a computer platform that enables smart device users without prior property inspection training to perform property inspections without sacrificing quality. The disclosed technology also provides a computational platform that enables property inspections and / or loss reporting in a more time-efficient and economical manner compared to employing trained property inspectors. Insurance providers and other parties can rely on actual property data, rather than advanced analytics and assumptions. It also provides a computational platform that enables insurance providers and other parties to quote coverage more quickly, more accurately, and in a more personalized / tailored manner, ensuring appropriate pricing and coverage levels, and allowing insurers to accurately understand risk exposure and property value.

[0069] With respect to some of the embodiments shown above, it should be understood that the various non-limiting embodiments described herein can be used alone, in combination, or selectively combined for a particular application. Furthermore, some of the various features of the above non-limiting embodiments can be used without correspondingly using the other described features. Therefore, the above description should be considered merely as illustrating the principles, teachings, and exemplary embodiments of the invention, and not as limiting it.

[0070] It should be understood that the above arrangements merely illustrate the application of the principles of the illustrated embodiments. Many modifications and alternative arrangements can be devised by those skilled in the art without departing from the scope of the illustrated embodiments, and the appended claims are intended to cover such modifications and arrangements.

[0071] Cross-references to related applications

[0072] This application is a continuation-in-part application that claims priority to U.S. Patent Application Serial No. 18 / 509,423, filed November 15, 2023, pursuant to 35 U.S.C., Section 120, and was published on March 14, 2024, as U.S. Patent Application Publication No. US20240087061. That application claims priority to U.S. Patent Application Serial No. 18 / 045,861, filed October 12, 2022, and was published in December 2023. Published on the 26th under U.S. Patent No. 11,854,100, which claims priority to U.S. Patent Application Serial No. 16 / 276,405, filed on February 14, 2019, and on November 8, 2022 under U.S. Patent No. 11,494,857, which claims priority to U.S. Provisional Patent Application Serial No. 62 / 631,266, filed on February 15, 2018, the entire contents of which are incorporated herein by reference as if presented in full.

Claims

1. A computer-implemented method, comprising: Receive input commands at the mobile computing device to initiate the capture of audiovisual documents of the environment; Output from the mobile computing device: The user utters an instruction for one or more test phrases; as well as A user training progress indicator is configured to advance in response to auditory detection of one or more test phrases; During the training phase: Receive auditory input corresponding to the instruction; as well as The user training progress indicator is advanced in response to the received auditory input; as well as In response to receiving an auditory input of a predetermined threshold amount, switch to the audiovisual capture stage; During the audiovisual capture phase: The mobile computing device captures audiovisual documents, which include video of the environment and audio narration by the user.

2. The computer-implemented method according to claim 1 further includes transmitting the audiovisual document to a remote server.

3. The computer-implemented method according to claim 1, wherein, The audiovisual document is captured at least in part by the rear camera of the mobile computing device.

4. The computer-implemented method according to claim 1, wherein, The user training progress indicator is configured to provide visual or auditory feedback to the user to prompt the user to speak about the audiovisual document.

5. The computer-implemented method according to claim 1, wherein, The audiovisual documents include damage to the structure of the environment.

6. The computer-implemented method according to claim 1, wherein, The audiovisual materials include damage to the vehicle.

7. The computer-implemented method according to claim 1, wherein, The user training progress indicator includes one or more auditory and visual information.

8. The computer-implemented method of claim 1 further includes detecting one or more objects located in the environment based on rule groups, wherein, The rule set enables analysis to determine whether the one or more objects match one or more objects that are pre-existing in the environment, and whether one or more pre-defined specifications can be determined based on the analysis of the one or more objects.

9. The computer-implemented method of claim 1 further includes outputting user instructions from the mobile computing device to capture an audiovisual document of the environment.

10. A computer system for executing an audiovisual document of a capture environment, comprising: Mobile computing devices, including: The camera is configured to capture video; The microphone is configured to capture audio; One or more processors that communicate with the camera and the microphone; The first memory is configured to store the captured video and audio; A second memory stores computer code that causes the one or more processors to perform the following operations: Receive input commands to initiate the capture of audiovisual documents of the environment; Output from the mobile computing device: The user utters one or more test phrases; and A user training progress indicator is configured to advance in response to auditory detection of one or more test phrases; During the training phase; Receive auditory input corresponding to the instruction; and In response to the received auditory input, the user training progress indicator is advanced; and In response to receiving an auditory input of a predetermined threshold amount, switch to the audiovisual capture stage; During the audiovisual capture phase: The mobile computing device captures audiovisual documents, which include video of the environment and audio narration by the user.

11. The computer system of claim 10, further comprising a property inspection server system coupled to a network communicating with the mobile computing device and configured to generate a report based on the condition of one or more objects in the environment, the report including a loss estimate.

12. The computer system according to claim 10, wherein, The environment includes one or more of the following: kitchen, living room, bedroom, bathroom, garage, external structure, and roof.

13. The computer system of claim 10, further comprising a feature extractor system, the feature extractor system utilizing one or more of an artificial intelligence (AI) application programming interface and a set of rules to analyze one or more objects in the environment to determine whether the one or more objects match one or more objects pre-existing in the environment.

14. The computer system according to claim 10, wherein, The computer code also enables the one or more processors to transmit the audiovisual document to a remote server.

15. The computer system according to claim 10, wherein, The mobile computing device is configured to capture the audiovisual document at least in part by a rear-facing camera.

16. The computer system according to claim 10, wherein, The user training progress indicator is configured to provide visual or auditory feedback to the user to prompt the user to speak about the audiovisual document.

17. The computer system according to claim 10, wherein, The audiovisual documents include damage to the structure of the environment.

18. A non-transitory computer-readable storage medium storing computer code configured to cause one or more processors to perform methods including: Receive input commands at the mobile computing device to initiate the capture of audiovisual documents of the environment; Output from the mobile computing device: The user utters an instruction for one or more test phrases; as well as A user training progress indicator is configured to advance in response to auditory detection of one or more test phrases; During the training phase; Receive auditory input corresponding to the instruction; as well as The user training progress indicator is advanced in response to the received auditory input; as well as In response to receiving an auditory input of a predetermined threshold amount, switch to the audiovisual capture stage; During the audiovisual capture phase: The mobile computing device captures audiovisual documents, which include video of the environment and audio narration by the user.

19. The non-transitory computer-readable storage medium according to claim 18, wherein, The computer code is also configured to cause the one or more processors to output one or more visual and auditory feedbacks to the user to prompt the user to narrate the audiovisual document.

20. The non-transitory computer-readable storage medium according to claim 18, wherein, The computer code is also configured to cause the one or more processors to detect one or more objects in the environment based on a set of rules, wherein the set of rules enables analysis to determine whether the one or more objects match one or more objects pre-existing in the environment, and whether one or more loss specifications can be determined based on the analysis of the one or more objects.

Images