Operation method of electronic device for performing natural language processing and inference on digital forensic analysis result

Abstract

An operation method of an electronic device is disclosed. This operation method comprises the steps of: extracting text related to at least one evidence item in a report file including a forensic analysis result; on the basis of attributes of respective entities included in the extracted text, acquiring a knowledge graph defining a relationship between the entities; and, according to the acquired knowledge graph, generating and providing a sentence containing the relationship between the entities.

Description

Method of operation of an electronic device that performs natural language processing and inference on the results of digital forensic analysis

[0001] The present disclosure relates to an electronic device for processing digital forensic analysis results, and more particularly, to an electronic device for defining relationships between objects within evidence items through a knowledge graph and providing meaningful information through inference.

[0002] The digital forensics industry is a field that utilizes technology to recover / extract / analyze digital data from digital media with legal / technical integrity. The most basic service type is mounting / recovering / extracting / analyzing evidence image files, and representative solutions for this include EnCase (OpenText) and Axiom (Magnet Forensics).

[0003] The present disclosure provides an operating method of an electronic device that processes digital forensic analysis results, such as DBs, logs, and registries, that can be understood by experts in the form of a knowledge graph or sentences, so that even general investigators / investigators who are not digital forensic experts can easily understand the results, and in particular, enables causal inference in relation to content evidence extracted from other sources.

[0004] The purposes of the present disclosure are not limited to those mentioned above, and other purposes and advantages of the present disclosure not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present disclosure. Furthermore, it will be readily apparent that the purposes and advantages of the present disclosure can be realized by the means and combinations thereof set forth in the claims.

[0005] A method of operating an electronic device according to one embodiment of the present disclosure includes the steps of extracting text for at least one evidence item from a report file including a result of forensic analysis, obtaining a knowledge graph defining a relationship between objects based on properties of each object included in the extracted text, and generating and providing a sentence including a relationship between objects based on the obtained knowledge graph.

[0006] The method of operating the electronic device may include a step of generating inference data related to the evidence item based on the acquired knowledge graph, a step of updating the knowledge graph based on the generated inference data, and a step of generating and providing a sentence including a relationship between objects based on the updated knowledge graph.

[0007] The step of generating the above inference data may generate inference data for at least one attribute or other evidence item related to the entities included in the knowledge graph within the report file according to a preset rule based on the attributes of the entities and the relationships between the entities.

[0008] At this time, the step of generating the inference data may include a step of applying the preset rule to a first target evidence item among the plurality of evidence items to infer an attribute of at least one first main entity related to the crime among entities included in the first target evidence item, a step of applying the preset rule to the attribute of the inferred first main entity to select at least one second target evidence item related to the first target evidence item, and a step of applying the preset rule to the selected second target evidence item to infer an attribute of at least one second main entity related to the crime among entities included in the second target evidence item.

[0009] In this case, the step of generating the inference data may include, when a user input requesting a search for the second primary entity is received, a step of selecting at least one third target evidence item related to the second primary entity, and a step of applying the preset rule to the selected third target evidence item to infer an attribute of at least one third primary entity related to the crime among the entities included in the third target evidence item.

[0010] The step of generating the above inference data may also generate the inference data by inputting information about the entities constituting the acquired knowledge graph into an artificial intelligence model trained to perform inference based on the properties of entities constituting multiple evidence items related to various crimes, relationships between entities, and the content of the crime.

[0011] Meanwhile, the operating method of the electronic device may include a step of classifying a plurality of evidence items included in the report file by category and setting them into a plurality of nodes; a step of classifying evidence items included in the node when a user input requesting classification by subject or time for at least one node among the plurality of nodes is received; a step of selecting at least one fourth target evidence item by filtering the evidence items included in the node according to a keyword selected according to the user input; and a step of generating and providing a sentence about a criminal relationship matching the keyword based on a relationship between objects in a text constituting the selected fourth target evidence item.

[0012] The method of operating an electronic device according to the present disclosure provides not simply interpreted fields for a report of a forensic analysis result, but sentences in a natural language form with added predicate-type relationships and knowledge graph data which is data that generated the natural language, thereby enabling even non-experts to intuitively interpret data, and by providing it in a SUBJECT-OBJECT-PREDICATE (SOP) form that enables causal relationship inference rather than a simple listed DB form, the utility of digital evidence in investigations / investigations can be maximized.

[0013] FIG. 1 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure;

[0014] FIG. 2 is a flowchart illustrating the operation of an electronic device that visualizes evidence items according to a knowledge graph and provides them as natural language processed sentences according to one embodiment of the present disclosure;

[0015] FIG. 3 is a diagram illustrating the configuration of a typical report including evidence items analyzed by an electronic device according to one embodiment of the present disclosure;

[0016] FIG. 4 is a diagram for explaining an operation of an electronic device according to an embodiment of the present disclosure to identify relationships between objects according to preset rules and visualize them as a knowledge graph.

[0017] FIG. 5A is a diagram illustrating an operation of an electronic device according to an embodiment of the present disclosure to classify and provide evidence items by category;

[0018] FIG. 5b is a diagram for explaining an operation of an electronic device according to an embodiment of the present disclosure to provide a knowledge graph by visualizing each attribute of an object included in each evidence item;

[0019] FIG. 6 is a flowchart illustrating an operation of an electronic device according to an embodiment of the present disclosure to perform inference based on a knowledge graph for an evidence item and to update and provide the knowledge graph according to the result of the inference.

[0020] FIG. 7 is a diagram for explaining an operation of an electronic device according to an embodiment of the present disclosure to infer the meaning of each object constituting a natural language through a dictionary DB pre-stored by object attribute.

[0021] FIGS. 8A to 8D are diagrams illustrating an operation of an electronic device according to an embodiment of the present disclosure to support a user's in-depth evidence analysis of a plurality of sequential evidence items while providing inference data for the evidence items.

[0022] FIGS. 9A and 9B are drawings for explaining an operation of an electronic device according to an embodiment of the present disclosure to classify evidence items by category while providing a filtering function according to keywords; and

[0023] FIG. 10 is a block diagram illustrating a detailed configuration of an electronic device according to various embodiments of the present disclosure.

[0024] Before describing the present disclosure in detail, the description method of the specification and drawings will be described.

[0025] First, the terms used in this specification and claims are general terms selected based on their functions in the various embodiments of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, legal or technical interpretations, and the emergence of new technologies. Furthermore, some terms may have been arbitrarily selected by the applicant. These terms may be interpreted according to the meanings defined in this specification. In the absence of a specific definition, they may be interpreted based on the overall content of this specification and common technical knowledge in the relevant field.

[0026] Additionally, the same reference numbers or symbols in each drawing attached to this specification represent parts or components that perform substantially the same functions. For convenience of explanation and understanding, the same reference numbers or symbols are used in different embodiments. In other words, even if components with the same reference numbers are all depicted in multiple drawings, the multiple drawings do not necessarily represent a single embodiment.

[0027] Additionally, terms including ordinal numbers, such as "first," "second," etc., may be used in this specification and claims to distinguish between components. These ordinal numbers are used to distinguish identical or similar components from each other, and the use of these ordinal numbers should not be interpreted in a limited manner. For example, components associated with these ordinals should not be restricted in their order of use or arrangement by their numbers. If necessary, each ordinal number may be used interchangeably.

[0028] In this specification, singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0029] In the embodiments of the present disclosure, terms such as "module," "unit," "part," etc. are terms used to refer to components that perform at least one function or operation, and such components may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of "modules," "units," "parts," etc. may be integrated into at least one module or chip and implemented as at least one processor, except in cases where each needs to be implemented as a separate, specific hardware.

[0030] Additionally, in the embodiments of the present disclosure, when a part is said to be connected to another part, this includes not only a direct connection but also an indirect connection through another medium. Furthermore, unless specifically stated otherwise, the statement that a part includes a certain component does not exclude other components, but rather implies that other components may be included.

[0031] FIG. 1 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0032] Referring to FIG. 1, an electronic device (100) may include a memory (110) and a processor (120).

[0033]

[0034] *The electronic device (100) may be implemented as a server including at least one computer, or as a system including multiple server devices. The electronic device (100) implemented as a server may perform operations in conjunction with various user terminal devices through at least one web page or application.

[0035] Alternatively, the electronic device (100) may be implemented as various terminal devices such as a desktop PC, a laptop PC, a smartphone, or a tablet PC.

[0036] The memory (110) is a configuration for storing an operating system (OS) for controlling the overall operation of components of the electronic device (100) and at least one instruction or data related to the components of the electronic device (100).

[0037] The memory (110) may include non-volatile memory such as ROM, flash memory, etc., and may include volatile memory composed of DRAM, etc. In addition, the memory (110) may include a hard disk, SSD (Solid state drive), etc.

[0038] The processor (120) is a component for controlling the overall operation of the electronic device (100). Specifically, the processor (120) is connected to the memory (110) and executes at least one instruction stored in the memory (110) to perform operations according to various embodiments of the present disclosure.

[0039] The processor (120) may include a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-only processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. The artificial intelligence-only processor may be designed with a hardware structure specialized for training or utilizing a specific artificial intelligence model.

[0040] Referring to FIG. 1, the processor (120) can control various modules, such as a text extraction module (121), an object relationship setting module (122), a visualization module (123), a text generation module (124), and an inference module (125). Each of these modules corresponds to a functional unit implemented in software and / or hardware.

[0041] The text extraction module (121) is a module for extracting text constituting each evidence item within a report file that is the result of forensic analysis.

[0042] A report file may be, for example, a report on forensic evidence provided by EnCase, Axiom, etc., but may also correspond to various other formats of reports containing data on forensic evidence of an electronic device as evidence. The report file may be selected and uploaded based on user input. For example, when software / applications configured with respective functions corresponding to operations according to various embodiments of the present disclosure are running on an electronic device (100), a user input for uploading at least one report file on the software may be received.

[0043] If the report file is in image format, the text extraction module (121) can extract the text constituting each evidence item in a manner such as OCR (Optical Character Recognition).

[0044] The entity relationship setting module (122) is a module for identifying the properties of each entity in the text constituting the evidence item and setting the relationship between the entities.

[0045] The object relationship setting module (122) can perform natural language understanding on text, convert the text into vector form, and identify the properties of each object.

[0046] Specifically, if the source, location, evidence number, evidence name, URL, website name, etc. of each evidence item are recorded according to the basic classification within the report file, the entity relationship setting module (122) can recognize the entity according to the basic classification. However, if the contents of each evidence item within the report file are not organized according to the basic classification, the entity relationship setting module (122) can perform vector conversion through natural language understanding of the text of each evidence item, and recognize multiple entities by dividing the converted vector into word units.

[0047] At this time, the object relationship setting module (122) can obtain a knowledge graph that defines the relationship between objects according to the properties of each object.

[0048] Attributes of an entity may include the basic classification of data matching the entity (as defined by the report), the meaning of the entity, the nature of the entity in relation to the crime (e.g., subject, action, object, violation law, etc.), and the attributes of data associated with the entity (e.g., deletion, message, log, time, etc.).

[0049] The entity relationship setting module (122) can define relationships between entities based on the properties of each entity included in an evidence item and preset rules. At this time, the preset rules can include information on relationships between entities defined according to the properties of each entity included in the same evidence item. In addition, the preset rules can include information on relationships between entities defined according to the properties of each entity included in different evidence items.

[0050] For example, relationships between entities can be identified based on attributes related to the processing of data within an evidence item (e.g., action, source, timestamp, etc.). For example, for one evidence item, if the action is "delete," the timestamp is "2024.01.04 19:16," and the file name is "ABC," then based on the above-described rule, the file name "ABC" can be identified as having been deleted on 2024.01.01 19:16.

[0051] For example, within the evidence items corresponding to the message content, if the account "Kim Cheol-su" sent a message such as "For sale of Philopon," "Kim Cheol-su," "Philopon," and "for sale" could each be recognized as separate entities. In this case, the sender of the message, which includes both the entity "Philopon," which refers to drugs, and the entity "For sale," which refers to sales, can be identified as not only the sender of the message but also the seller of drugs, according to the aforementioned rules.

[0052] The visualization module (123) can generate a knowledge graph based on the attributes of each entity and the relationships between entities described above. For example, the terms "Kim Cheol-su," "philopon," and "sales" described above may be directly connected (e.g., by a line) within the knowledge graph. In this case, information regarding the attributes and relationships of each entity may be included within the knowledge graph.

[0053] In this case, the visualization module (123) can provide an image representing the generated knowledge graph. If the electronic device (100) is a server, the electronic device (100) can provide the image through a user's terminal device (e.g., a smartphone, tablet PC, desktop PC, etc.) that has accessed a web page or application provided by the electronic device (100). If the electronic device (100) is a user's terminal device, the electronic device (100) can output the image through a display provided by the electronic device (100).

[0054] The text generation module (124) is a module for generating sentences based on the attributes and relationships of objects identified by the object relationship setting module (122). The generated sentences can also be provided visually, similar to a knowledge graph.

[0055] For example, sentences such as “Kim Cheol-su is a seller of methamphetamine” and “File ABC was deleted on 2024.01.01 19:16” can be generated and provided.

[0056] To this end, the text generation module (124) may include a natural language generation module trained to generate sentences based on the properties and relationships of each entity identified by the entity relationship setting module (122) and / or trained to generate sentences based on a knowledge graph generated through the visualization module (123). Specifically, the natural language generation module may include at least one neural network model trained to generate sentences based on the properties and relationships of each entity constituting the knowledge graph.

[0057] The inference module (125) is a module for generating inference data related to evidence items based on a knowledge graph. The inference module (125) may perform inference according to the above-described rules, or may perform inference using at least one artificial intelligence model trained to perform inference (e.g., artificial intelligence models related to natural language understanding / processing / generation, RNN (Recurrent Neural Network), Transformer, GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), etc.).

[0058] FIG. 2 is a flowchart illustrating the operation of an electronic device that visualizes evidence items according to a knowledge graph and provides them as sentences processed in natural language according to one embodiment of the present disclosure.

[0059] Referring to FIG. 2, the electronic device (100) can extract text for at least one evidence item within a report file containing the results of forensic analysis (S121).

[0060] Specifically, the text extraction module (121) can extract text constituting each of a plurality of evidence items included in a report file.

[0061] For example, referring to FIG. 3, the text extraction module (121) can identify text for each evidence item based on basic classifications such as source, location, evidence number, evidence name, URL, and website name. At this time, texts matching each basic classification can be identified as separate entities.

[0062] The electronic device (100) can obtain a knowledge graph that defines relationships between objects based on the properties of each object included in the extracted text (S220).

[0063] Specifically, the object relationship setting module (122) can define relationships between objects based on the properties of each object included in the evidence item and preset rules.

[0064] At this time, the visualization module (123) can generate a knowledge graph based on the relationships between objects, and the knowledge graph can be provided to the user in the form of an image. The knowledge graph corresponds to a diagram that includes information about the content, properties, and relationships between each object.

[0065] If the electronic device (100) is a server, the visualization module (123) can communicate with the user's terminal device and control the display of the terminal device to display an image of the knowledge graph. If the electronic device (100) is the user's terminal device, the visualization module (123) can display the knowledge graph through the display of the electronic device (100).

[0066] For example, as shown in Fig. 4, a connection relationship between objects within a knowledge graph can be derived based on the relationship between objects, and at least one of the size, shape, and color of a UI item (e.g., circle, triangle, square, star, etc.) representing each object can be changed based on the properties of each object.

[0067] In addition, the electronic device (100) can generate and provide a sentence including a relationship between objects according to the acquired knowledge graph (S230).

[0068] Specifically, the text generation module (124) can generate sentences based on a knowledge graph. For example, a sentence such as "I accessed https: / vonpat.net on 2023.01.03 10:11" can be generated based on the properties and relationships of each entity such as "2023.01.03 10:11" (time), "https: / vonpat.net" (web address), and "access" (action).

[0069] Meanwhile, FIG. 5A is a diagram for explaining an operation of an electronic device according to an embodiment of the present disclosure to classify and provide evidence items by category.

[0070] Referring to FIG. 5a, the electronic device (100) can identify multiple evidence items by analyzing a report file containing the forensic analysis results for the devices (phone, PC) of the target person named “Hong Gil-dong.”

[0071] At this time, the electronic device (100) can classify multiple evidence items included in the report file into multiple nodes by category. In addition to upper categories such as Media, DB, and Log, the categories can include lower categories such as Messenger, Web History, and Registry.

[0072] In relation to this, FIG. 5b is a diagram for explaining an operation of an electronic device according to an embodiment of the present disclosure to provide a knowledge graph by visualizing it according to the properties of each object included in each evidence item.

[0073] Referring to FIG. 5b, the electronic device (100) can obtain and visualize a knowledge graph for evidence items included in at least one node (e.g., Registry, Web History) as in the above-described embodiment and provide it.

[0074] For example, referring to FIG. 5b, for the evidence item included in the Registry node, the object “HKEY_CURRENT_USER / Software / ccleaner” was identified by the key_name (attribute), the object “2023.01.04 19:16:09” was identified by the related time attribute (last_used_time), and a knowledge graph showing the attributes and relationships of each object was displayed as in FIG. 5b.

[0075] In addition, referring to Fig. 5b, for the object “http: / hgd77i3xxxx.onion” corresponding to the URL (attribute), the objects corresponding to each attribute of source, action, and last_visit_time are visualized and provided in the form of a knowledge graph.

[0076] The knowledge graph visualized through the processes of FIGS. 5A and 5B can be displayed via an electronic device (100) or a terminal device connected to the electronic device (100). As a result, the user can easily grasp the properties and relationships of each entity constituting the evidence item at a glance.

[0077] Meanwhile, FIG. 6 is a flowchart illustrating an operation in which an electronic device according to an embodiment of the present disclosure performs inference based on a knowledge graph for an evidence item and updates and provides the knowledge graph according to the result of the inference.

[0078] Referring to FIG. 6, the electronic device (100) can generate inference data related to an evidence item based on a knowledge graph acquired for the evidence item according to at least one of the embodiments described above (S610).

[0079] Specifically, the inference module (125) can generate inference data for at least one attribute or other evidence item related to entities included in the knowledge graph according to the above-described rules set based on the relationships between entities.

[0080] And, when inference data is generated according to at least one of the various embodiments described above, the electronic device (100) can update the knowledge graph based on the generated inference data (S620).

[0081] As a result, at least one entity, entity property, entity relationship, etc. that constitutes inference data can be added to the knowledge graph constructed based on the properties and relationships of entities that constitute existing evidence items.

[0082] As an example of an embodiment related to S610, the inference module (125) can identify the properties of each object through at least one dictionary DB for identifying what each object means.

[0083] In relation to this, FIG. 7 is a diagram for explaining an operation of an electronic device according to an embodiment of the present disclosure to infer the meaning of each object constituting a natural language through a dictionary DB pre-stored by object attribute.

[0084] Referring to FIG. 7, the electronic device (100) can acquire text (710) for each evidence item (file name) corresponding to messages exchanged online by the target person “Hong Gil-dong,” and each text can be composed of multiple objects. At this time, the electronic device (100) can identify the properties of each object according to a dictionary DB (720) classified by object properties. The dictionary DB (720) can be classified and pre-stored according to the properties of various objects, such as a person dictionary, a place dictionary, a domain dictionary, and a verb dictionary.

[0085] For example, if "Im Kkeok-jeong" sends a message saying "I sell ice," the entities "ice" and "panda" can be identified within the text, and at this time, the inference module (125) can identify "ice" as "philopon" based on the domain dictionary. In this case, the inference module (125) can identify "Im Kkeok-jeong" as a drug seller.

[0086] Additionally, as an example, the inference module (125) can generate inference data by applying the above-described rules to the properties and relationships of objects included in each evidence item.

[0087] In relation to this, FIGS. 8A to 8D are drawings for explaining an operation of an electronic device according to an embodiment of the present disclosure to support a user's in-depth evidence analysis of a plurality of sequential evidence items while providing inference data for the evidence items.

[0088] Figure 8a assumes that multiple evidence items are categorized and included in each node, as in Figure 5a described above. In this case, the inference module (125) can generate inference data for each of the multiple evidence items included in the report file according to the above-described rules.

[0089] Alternatively, the inference module (125) may set at least one evidence item selected according to user input as a target evidence item and generate inference data only for the target evidence item.

[0090] For example, referring to FIG. 8A, the inference module (125) can identify at least one key object related to a crime among the objects included in the target evidence item (811) by applying a preset rule to at least one target evidence item (811) included in the Registry. Specifically, in the target evidence item (811), an object corresponding to key_name “HKEY_CURRENT_USER / Software / ccleaner” (811-1) and corresponding to last_used_time “2023.01.04 19:16:09” (811-2) can be identified. At this time, since “ccleaner” included in “HKEY_CURRENT_USER / Software / ccleaner” corresponds to a function for deletion, the object (811-1) can be identified as a key object.

[0091] In this case, the inference module (125) can perform inference on the main object according to the above-described rule. Specifically, the attribute (action: delete) of 'ccleaner' can be inferred. At this time, the inference module (125) can obtain inference data (811') including the action and timestamp corresponding to the deletion. In other words, the time at which the deletion function was utilized can be inferred.

[0092] As a result, a knowledge graph can be provided in the form of each object that constitutes the inference data (811') as shown in Fig. 8a.

[0093] In addition, the inference module (125) can infer and select at least one other target evidence item based on the properties of the main object (811-1) of the target evidence item (811).

[0094] Specifically, referring to FIG. 8a, the inference module (125) can identify a target evidence item (812) for which an action related to deletion has been performed for a certain period of time (e.g., 1 day) according to a rule from the time when the deletion function of the target evidence item (811) has been performed.

[0095] At this time, the inference module (125) can identify the URL “http: / hgd77i3xxxx.onion” related to the crime as the main object among the objects constituting the target evidence item (812). To this end, the inference module (125) can utilize a malicious URL DB containing addresses of dark web, illegal sites, etc.

[0096] At this time, the inference module (125) can identify the properties of the URL, which is the main object, as hidden information, a suspicious URL, a dark web, etc., and can add and display inference data (812') including the inferred properties to the knowledge graph, as shown in FIG. 8b.

[0097] Meanwhile, the electronic device (100) can perform a search for a key object included in at least one target evidence item. Specifically, the electronic device (100) can search for at least one other target evidence item related to the key object.

[0098] In relation to this, referring to FIG. 8c, the electronic device (100) can perform a search for the main object “http: / hgd77i3xxxx.onion” URL described above in FIG. 8b through a UI (User Interface) item (850) for search execution.

[0099] Based on the search results, the electronic device (100) can identify a target evidence item (813) that includes the URL as an object. At this time, the electronic device (100) can perform inference by applying the target evidence item (812) and the target evidence item (813) to the above-described rules.

[0100] As a result, additional inference can be performed on the target evidence item (813), as shown in FIG. 8d.

[0101] At this time, a user input requesting OSINT analysis for the entity "iceice5" included in the target evidence item (813) corresponding to the name may be received. In this case, a search of public websites or social media may be performed. For example, in relation to "iceice5," a tag such as "ice sale" may be derived, and based on the database of FIG. 7 described above, "ice sale" may be inferred to mean "philopon sale."

[0102] As a result, the inference module (125) can infer that the attribute of “iceice5” is a drug seller (supplier identity), and can infer that the sender “Im Kkeok-jeong,” an entity included in the target evidence item (813), is a drug seller. Furthermore, as shown in FIG. 8d, inference data (813') including each inferred entity and attribute can be added to the knowledge graph and displayed.

[0103] Meanwhile, in the process of generating inference data, the inference module (125) may utilize at least one artificial intelligence model. The artificial intelligence model may correspond to a neural network model stored in the memory (110), and may be composed of a Recurrent Neural Network (RNN), a Transformer, a Bidirectional Encoder Representations from Transformers (BERT), a Generative Pre-trained Transformer (GPT), etc. for performing language (entity)-based inference, but is not limited thereto.

[0104] The inference module (125) can input information about entities constituting the knowledge graph into the artificial intelligence model to generate inference data. For example, the inference process illustrated in FIGS. 7 and 8a to 8d described above can be performed using an artificial intelligence model that is not rule-based.

[0105] This AI model can be trained to make inferences based on the attributes of entities comprising multiple evidence items related to various crimes, the relationships between entities, and the nature of the crime. Specifically, the AI ​​model can select at least one key entity among the entities included in the evidence items that may be related to the crime and infer the attributes of the selected key entity. Furthermore, the model can infer and select at least one other evidence item based on the attributes of the key entity.

[0106] In relation to this, an electronic device (100) according to an embodiment of the present disclosure may update an artificial intelligence model according to user input for inference data generated by an artificial intelligence model for inference.

[0107] In one embodiment, the electronic device (100) may input information (e.g., a knowledge graph) regarding entities constituting each of a plurality of evidence items included in a report file into an artificial intelligence model. In this case, inference data may not be generated for some evidence items, but for at least one evidence item (target evidence item), inference data related to at least one entity constituting the corresponding evidence item may be generated.

[0108] At this time, the inference data may include attributes of an entity constituting the target evidence item, or may include at least one other evidence item related to at least one entity constituting the target evidence item (e.g., a primary entity).

[0109] In this case, the electronic device (100) can provide the user with information about target evidence items and inference data. At this time, the information may be provided in the form of a knowledge graph, as shown in FIGS. 8A to 8C , or a sentence generated based on the knowledge graph may be provided.

[0110] Additionally, the electronic device (100) may receive user input requesting additional inference or retrieval of the inference data.

[0111] When a user's inference request is received for the generated inference data, the electronic device (100) can input a target knowledge graph including existing target evidence items and inference data into the artificial intelligence model.

[0112] If additional inference data for the AI ​​model for the target knowledge graph is acquired, the electronic device (100) can train the AI ​​model based on the knowledge graph containing the initially acquired inference data and the additional inference data. Thereafter, the inference of the AI ​​model, previously performed through two steps (initial inference and inference based on an inference request), can be improved to be performed in a single step.

[0113] When a user's search request for a search term consisting of at least a portion of generated inference data is received, the electronic device (100) can identify and provide at least one other evidence item related to the search term among a plurality of evidence items.

[0114] In this way, when different evidence items are identified based on search results, the electronic device (100) can train an artificial intelligence model based on a knowledge graph that includes existing target evidence items, inference data, and evidence items corresponding to the search results. Subsequently, the system can be improved to infer the relationship between the evidence items matching the search results and the target evidence items in a single step, without requiring additional search requests from the user.

[0115] When inference data for at least one evidence item is generated according to at least one of the various embodiments described above, and a knowledge graph is updated according to the generated inference data, the electronic device (100) can generate and provide a sentence including a relationship between objects according to the updated knowledge graph.

[0116] For example, if the original evidence item was simply provided as the sentence “iceice5 sold ice,” as inference data is added, it can be provided as the sentence “Im Kkeok-jeong sold Philopon.”

[0117] Meanwhile, FIGS. 9A and 9B are drawings for explaining an operation of an electronic device according to an embodiment of the present disclosure to classify evidence items by category while providing a filtering function according to keywords.

[0118] Referring to FIG. 9a, in a plurality of nodes (e.g., Media, DB, Log, Messenger, Web History, etc.) where multiple evidence items are classified by category, the electronic device (100) can select at least one node based on user input. In the case of FIG. 9a, the Messenger node, which is a sub-node of the DB, is selected.

[0119] Here, when a user input requesting a classification by topic or time for a selected node is received, the electronic device (100) can classify the evidence items included in the node.

[0120] To this end, the electronic device (100) may provide respective UI items (911, 912) for subject-based grouping and time-based grouping, as shown in FIG. 9a.

[0121] For example, when a UI item (911) for topic-based grouping is selected, the electronic device (100) can classify each evidence item (e.g., each user's message transmission and reception history) by topic, such as daily life, advertisement, business, etc., as shown in FIG. 9a. To this end, for each topic, the degree of association of words corresponding to entities included in each evidence item is calculated, so that each evidence item can be classified to be included in the topic with the highest degree of association. For example, in various disclosed digital files, the degree of association can be calculated to be higher as the frequency of co-occurrence of each entity for words corresponding to the topic increases, but is not limited thereto.

[0122] At this time, referring to FIG. 9b, the electronic device (100) can provide a UI (920) for filtering evidence items, and for example, when a user input requesting a keyword with “drugs” as a keyword is received, the electronic device (100) can select evidence items (921) including objects related to drugs (e.g., ice, philopon, tremors, etc.).

[0123] In this case, based on a user input requesting inference, the inference module (125) can generate and provide a sentence about a criminal relationship matching a keyword based on the relationship between objects constituting each of the selected evidence items (921).

[0124] For example, a sentence like "Igumae purchased Philopon" can be generated based on the relationships between entities like "Igumae," "Ice," and "Purchase." Similarly, sentences like "Choigumae purchased Philopon" and "Igumae and Choigumae jointly purchased Philopon" can be generated and provided based on the relationships between entities included in one or more evidence items (e.g., a knowledge graph).

[0125] Meanwhile, FIG. 10 is a block diagram illustrating a detailed configuration of an electronic device according to various embodiments of the present disclosure.

[0126] Referring to FIG. 10, the electronic device (100) may include, in addition to a memory (110) and a processor (120), a communication unit (130), a user input unit (140), an output unit (150), etc.

[0127] The communication unit (130) may include circuits, modules, chips, etc. for performing communication with at least one external device using various wired and wireless communication methods.

[0128] The communication unit (130) can be connected to external devices through various networks.

[0129] Depending on the area or scale, a network may be a personal area network (PAN), a local area network (LAN), or a wide area network (WAN), and depending on the openness of the network, it may be an intranet, an extranet, or the Internet.

[0130] The communication unit (130) can be connected to external devices through various wireless communication methods such as LTE (long-term evolution), LTE-A (LTE Advance), 5G (5th Generation) mobile communication, CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), DMA (Time Division Multiple Access), WiFi (Wi-Fi), WiFi Direct, Bluetooth, NFC (near field communication), Zigbee, etc.

[0131] Additionally, the communication unit (130) can be connected to external devices through various wired communication methods such as Ethernet, Internet, optical communication, Thunderbolt, HDMI, and USB.

[0132] As an example, the electronic device (100) can be connected to a server / system of various platforms that provides a report file corresponding to forensic evidence through a communication unit (130) to obtain the report file.

[0133] When the electronic device (100) is a server, the electronic device (100) can be linked with various user terminal devices through the communication unit (130) to provide applications, web pages, etc.

[0134] When the electronic device (100) is implemented as a user's terminal device, the electronic device (100) may include a user input unit (140), an output unit (150), etc.

[0135] The user input unit (140) is configured to receive various user commands or information. The user input unit (140) may include at least one button, a touch pad, a microphone, a camera (for facial recognition, motion sensing), etc. Meanwhile, the electronic device (100) may communicate with at least one user input device (e.g., a remote controller, a smartphone, a mouse, a keyboard device) equipped with various user input means, and may also obtain information about user input from the user input device.

[0136] The output unit (150) is configured to visually / audibly output various information. The output unit (150) may include a display, LED, speaker, etc. Meanwhile, the electronic device (100) may output information through the output device by performing communication with at least one output device equipped with an output means such as a display or speaker.

[0137] For example, the electronic device (100) can provide text, a knowledge graph, sentences generated according to the knowledge graph, etc. that constitute a report file through the output unit (150), and can provide various UIs or UI items provided in the process of performing the various embodiments described above.

[0138] Meanwhile, the various embodiments described above may be implemented by combining two or more embodiments as long as they do not conflict or contradict each other.

[0139] Meanwhile, the various embodiments described above may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

[0140] In terms of hardware implementation, the embodiments described in the present disclosure may be implemented using at least one of Application Specific Integrated Circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0141] In some cases, the embodiments described herein may be implemented within the processor itself. In a software implementation, the embodiments described herein, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules described above may perform one or more of the functions and operations described herein.

[0142] Meanwhile, computer instructions or computer programs for performing processing operations in electronic devices according to the various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When executed by a processor of a specific device, the computer instructions or computer programs stored in the non-transitory computer-readable medium cause the specific device to perform the processing operations in the electronic device according to the various embodiments described above.

[0143] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0144] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In the method of operating an electronic device, A step of extracting text for at least one evidence item within a report file containing the results of the forensic analysis; A step of obtaining a knowledge graph defining relationships between objects based on the properties of each object included in the extracted text; and An operating method of an electronic device, comprising: a step of generating and providing a sentence including a relationship between objects according to the acquired knowledge graph; 2. In paragraph 1, The method of operation of the above electronic device is: A step of generating inference data related to the evidence item based on the acquired knowledge graph; A step of updating the knowledge graph based on the generated inference data; A method of operating an electronic device, comprising: a step of generating and providing a sentence including relationships between objects according to the updated knowledge graph.

3. In paragraph 2, The steps for generating the above inference data are: A method of operating an electronic device, wherein, based on a preset rule based on properties of objects and relationships between objects, inference data for at least one property or other evidence item related to objects included in the knowledge graph within the report file is generated.

4. In paragraph 3, The steps for generating the above inference data are: A step of applying the preset rule to a first target evidence item among the plurality of evidence items to infer attributes of at least one first main entity related to the crime among the entities included in the first target evidence item; A step of selecting at least one second target evidence item related to the first target evidence item by applying the preset rule to the properties of the first primary entity inferred above; and An operating method of an electronic device, comprising: a step of applying the preset rule to the selected second target evidence item to infer attributes of at least one second key object related to the crime among the objects included in the second target evidence item; 5. In paragraph 4, The steps for generating the above inference data are: When a user input requesting a search for said second primary entity is received, selecting at least one third target evidence item related to said second primary entity; and An operating method of an electronic device, comprising: a step of applying the preset rule to the selected third target evidence item to infer attributes of at least one third key object related to the crime among the objects included in the third target evidence item; 6. In paragraph 2, The steps for generating the above inference data are: An operating method of an electronic device, wherein information about entities constituting the acquired knowledge graph is input into an artificial intelligence model trained to perform inference based on the properties of entities constituting multiple evidence items related to various crimes, relationships between entities, and the content of the crime, thereby generating inference data.

7. In paragraph 1, The method of operation of the above electronic device is: A step of categorizing multiple evidence items included in the above report file and setting them into multiple nodes; When a user input requesting a classification by topic or time is received for at least one node among the plurality of nodes, a step of classifying evidence items included in the node; A step of selecting at least one fourth target evidence item by performing filtering on the evidence items included in the node according to a keyword selected according to user input; and A method of operating an electronic device, comprising: a step of providing a sentence about a criminal relationship matching the keyword based on relationships between objects in the text constituting the selected fourth target evidence item; 8. A computer-readable medium storing at least one instruction that is executed by a processor of an electronic device and causes the electronic device to perform the operating method of claim 1.

Images