Electronic device and control method of electronic device

Abstract

A control method of an electronic device is provided. The control method includes: obtaining a first eigenvalue corresponding to input data, mapping information included in the first eigenvalue to a symbol value corresponding to each of logically certain information and logically uncertain information, adjusting a gradient corresponding to the uncertain information, obtaining a second eigenvalue in which the first eigenvalue and the symbol value are combined, obtaining temporary answer information including the certain information and the uncertain information, obtaining score information related to the degree to which the temporary answer information matches a predefined logical rule, when the score information is less than a preset threshold, training a neural network model based on the adjusted gradient, and when the score information is greater than or equal to the preset threshold, obtaining output data about the input data based on the obtained second eigenvalue.

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a control method of the electronic device. More specifically, the present disclosure relates to an electronic device and a control method of the electronic device for applying a logical constraint to a neural network model. Background Art

[0002] Recently, with the application of technology in various fields, the technology of neural network models such as deep neural network (DNN) is constantly developing. However, when using neural network models, although high-probability results can be statistically / probabilistically derived by training the neural network model based on learning data, there are limitations in solving problems that require logical reasoning.

[0003] Furthermore, regarding the neural network models to date, safety issues have been raised, namely, if the training of the neural network model is based on biased or small amounts of learning data, it may lead to unethical results or results that exceed social common sense / norms.

[0004] In addition to the above, in order to demonstrate performance that satisfies the neural network model, a sufficient amount of learning data and computing resources as well as a long training process may be required, but there are also problems such as sufficient learning data, training time and computing resources cannot always be guaranteed in actual environments.

[0005] Therefore, there is a need for a technology that expands the usability of neural network models and trains efficiency by applying logical constraints to neural network models, so that the neural network models can not only output statistically high-probability results but also logically reasonable results.

[0006] The above information is presented as background information only to assist in understanding the present disclosure. No determination has been made, and no assertion has been made, as to whether any of the above might be applicable as prior art to the present disclosure. Summary of the invention

[0007] Technical issues

[0008] Aspects of the present disclosure at least solve the above-mentioned problems and / or disadvantages and provide at least the advantages described below. Therefore, one aspect of the present disclosure is to provide an electronic device and a control method of the electronic device, which expands the usability and learning efficiency of a neural network model by minimizing changes to the structure of an existing neural network model while applying logical rules, solves security issues regarding the neural network model by providing logically reasonable results for exceptional cases that cannot be solved by statistical determination based only on learning data, and minimizes unnecessary processing in the learning process of the neural network model by distinguishing statistical information from logically determined information.

[0009] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0010] Technical Solutions

[0011] According to one aspect of the present disclosure, an electronic device configured to apply a logical constraint to a neural network model is provided. The electronic device includes: a memory configured to store a neural network model, a symbol value mapping module, a merging module, a temporary answer acquisition module, a rule checking module, and a training determination module; and a processor configured to control the electronic device by executing at least one instruction stored in the memory. The processor is configured to: input the input data into the neural network model, obtain a first eigenvalue corresponding to the input data through a feature extraction module included in the neural network model; input the first eigenvalue into a symbol value mapping module; map the information included in the first eigenvalue to a symbol value corresponding to each of the logically certain information and the logically uncertain information; adjust the gradient corresponding to the uncertain information through the symbol value mapping module so that the update of the uncertain information becomes possible; obtain a second eigenvalue in which the first eigenvalue and the symbol value are merged by inputting the first eigenvalue and the symbol value into a merging module; obtain temporary answer information including certain information and uncertain information by inputting the first eigenvalue and the symbol value into a temporary answer acquisition module; obtain score information related to the degree to which the temporary answer information matches a predefined logical rule by inputting the temporary answer information into a rule checking module; based on the score information being less than a preset threshold, train the neural network model based on the adjusted gradient through a training determination module; and based on the score information being greater than or equal to the preset threshold, obtain output data about the input data by inputting the obtained second eigenvalue into a classification module included in the neural network model.

[0012] According to another aspect of the present disclosure, a control method of an electronic device is provided, the electronic device being configured to apply a logical constraint to a neural network model. The control method includes: based on input data input to a neural network model, obtaining a first feature value corresponding to the input data through a feature extraction module included in the neural network model; by inputting the first feature value to a symbol value mapping module included in the electronic device, mapping the information included in the first feature value to a symbol value corresponding to each of logically certain information and logically uncertain information; adjusting the gradient corresponding to the uncertain information through the symbol value mapping module so that updating of the uncertain information becomes possible; by inputting the first feature value and the symbol value to a merging module included in the electronic device, obtaining a second feature value in which the first feature value and the symbol value are merged; by inputting the first feature value and the symbol value to a temporary answer acquisition module included in the electronic device to obtain temporary answer information including certain information and uncertain information; by inputting the temporary answer information to a rule checking module included in the electronic device to obtain score information related to the degree to which the temporary answer information matches a predefined logical rule; and based on the score information being less than a preset threshold, training the neural network model based on the adjusted gradient through a training determination module included in the electronic device, and based on the score information being greater than or equal to the preset threshold, inputting the obtained second feature value to a classification module included in the neural network model to output data about the input data.

[0013] According to another aspect of the present disclosure, there is provided a computer-readable recording medium including a program for executing a control method of an electronic device. The control method of the electronic device includes: based on input data input to a neural network model, obtaining a first feature value corresponding to the input data through a feature extraction module included in the neural network model; by inputting the first feature value to a symbol value mapping module included in the electronic device, mapping the information included in the first feature value to a symbol value corresponding to each of logically certain information and logically uncertain information; adjusting the gradient corresponding to the uncertain information through the symbol value mapping module so that updating of the uncertain information becomes possible; by inputting the first feature value and the symbol value to a merging module included in the electronic device, obtaining a second feature value in which the first feature value and the symbol value are merged; by inputting the first feature value and the symbol value to a temporary answer acquisition module included in the electronic device to obtain temporary answer information including certain information and uncertain information; by inputting the temporary answer information to a rule checking module included in the electronic device to obtain score information related to the degree to which the temporary answer information matches a predefined logical rule; and based on the score information being less than a preset threshold, training the neural network model based on the adjusted gradient through a training determination module included in the electronic device, and based on the score information being greater than or equal to the preset threshold, inputting the obtained second feature value to a classification module included in the neural network model to output data about the input data.

[0014] Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a flowchart illustrating each operation of a control method of an electronic device according to an embodiment of the present disclosure;

[0017] Figure 2 is a flowchart illustrating a control method of an electronic device according to an embodiment of the present disclosure in more detail;

[0018] Figure 3 is a block diagram showing the architecture of a software module according to an embodiment of the present disclosure;

[0019] Figure 4 is a diagram illustrating in more detail the operation of a symbol value mapping module according to an embodiment of the present disclosure;

[0020] Figure 5is a diagram showing an embodiment of the present disclosure applied to solve a problem of selecting an operation under a given condition according to an embodiment of the present disclosure;

[0021] Figure 6 is a schematic diagram showing an embodiment of training a neural network model based on score information related to a degree of matching learning data according to an embodiment of the present disclosure;

[0022] Figure 7 is a diagram showing an embodiment of obtaining temporary answer information through a classification module of a neural network model according to an embodiment of the present disclosure;

[0023] Figure 8 is a block diagram briefly showing a hardware configuration of an electronic device according to an embodiment of the present disclosure; and

[0024] Fig. 9 is a block diagram illustrating in detail a hardware configuration of an electronic device according to an embodiment of the present disclosure.

[0025] Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures. DETAILED DESCRIPTION

[0026] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in understanding, but these are merely exemplary. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures may be omitted for clarity and conciseness.

[0027] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it is apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and not for limiting the present disclosure as defined by the appended claims and their equivalents.

[0028] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0029] Expressions used herein, such as but not limited to, "including", "may include", "contain", "may include", etc., may indicate the presence of corresponding features (for example, elements such as numbers, functions, operations, or components), and do not exclude the presence of additional features.

[0030] In the present disclosure, expressions such as "A or B", "at least one of A and / or B", or "one or more of A and / or B" may include all possible combinations of the items listed together. For example, "A or B", "at least one of A and B", or "at least one of A or B" may refer to all cases, including (1) at least one of A, (2) at least one of B, or (3) at least one of A and at least one of B.

[0031] Expressions such as “first,” “second,” “first,” “second,” etc. may be used for various elements regardless of order and / or importance, and are only used to distinguish one element from another without limiting the corresponding elements.

[0032] When an element (e.g., a first element) is expressed as being "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it can be understood that the specific element is directly coupled to the other element or coupled through yet another element (e.g., a third element).

[0033] On the other hand, when an element (for example, a first element) is expressed as being "directly coupled to / connected to" another element (for example, a second element) or "connected to" the other element (for example, the second element), it can be understood that there is no further element (for example, a third element) between the certain element and the other element.

[0034] The expression "configured to . . . (or arranged to)" used in this disclosure may be used interchangeably with, for example, "suitable for", "capable of", "designed to", "suitable for", "made to", or "capable of", depending on the context. The term "configured to . . . (or arranged to)" does not necessarily mean "specifically designed to" in terms of hardware.

[0035] More specifically, in some cases, the expression “the device is configured to…” may mean something that the device, together with another device or component, “can perform.” For example, the phrase “a processor configured to (or set to) perform A, B, and C” may represent a dedicated processor (e.g., an embedded processor) for performing the corresponding operations, or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) that can perform the corresponding operations by executing one or more software programs stored in a memory device.

[0036] In an embodiment, a 'module' or 'part' may perform at least one function or operation and may be implemented as hardware or software, or a combination of hardware and software. In addition, multiple 'modules' or multiple 'parts' may be integrated into at least one module and implemented in at least one processor, unless the 'module' or 'part' needs to be implemented in separate hardware.

[0037] Various elements and regions in the drawings have been schematically shown. Therefore, the technical idea of ​​the present disclosure is not limited by the relative sizes or distances shown in the drawings.

[0038] The electronic device according to various embodiments may include at least one of a smartphone, a tablet personal computer (PC), a desktop computer, a laptop PC, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head mounted device (HMD)), a fabric or clothing embedded type (e.g., an electronic garment), a body attached type (e.g., a skin pad or a tattoo), or a bio-implantable circuit.

[0039] In some embodiments, the electronic device may include at least one of, for example, but not limited to, a television, a digital video disc (DVD) player, a stereo, a refrigerator, a vacuum cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., SAMSUNG HOMESYNCTM, APPLE TVTM, or GOOGLETVTM), a game console (e.g., XBOXTM, PLAYSTATIONTM, etc.), an electronic dictionary, an electronic key, a portable camera, or an electronic photo frame.

[0040] In another embodiment, the electronic device may include at least one of various medical devices (e.g., various portable medical measuring devices (e.g., glucose measuring devices, heart rate measuring devices, blood pressure measuring devices, temperature measuring devices, etc.), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), imaging devices, ultrasonic devices, etc.), navigation equipment, global navigation satellite system (GNSS), event data recorder (EDR), flight data recorder (FDR), vehicle infotainment equipment, marine electronic equipment (e.g., marine navigation equipment, gyrocompass, etc.), avionics equipment, security equipment, vehicle host, industrial or personal robot, drone, automatic teller machine (ATM) of financial institution, point of sale (POS) of store or Internet of Things device (e.g., light bulbs, various sensors, sprinkler, fire alarm, thermostat, street lamp, toaster, exercise equipment, hot water tank, heater, boiler, etc.).

[0041] The embodiments will be described in more detail below with reference to the accompanying drawings to help those skilled in the art to easily understand the present disclosure.

[0042] Figure 1 is a flowchart showing each operation of a control method of an electronic device according to an embodiment of the present disclosure, and Figure 2 is a flowchart illustrating a control method of an electronic device according to an embodiment of the present disclosure in more detail.

[0043] Figure 3 is a block diagram illustrating the architecture of software modules according to an embodiment of the present disclosure.

[0044] The following will refer to Figures 1 to 3 Various embodiments of the present disclosure are described.

[0045] refer to Figure 1-3 , the 'electronic device' according to the present disclosure applies logical constraints to the neural network model, and as long as the device is configured to perform each stage of the control method described below, the device may correspond to the electronic device according to the present disclosure regardless of its type. For example, the electronic device may be implemented as various types such as a smartphone, a tablet PC, a notebook, and a digital television (TV), but the embodiment is not limited thereto. The electronic device according to the present disclosure may be described below as 'electronic device 100'.

[0046] The 'neural network model 1000' according to the present disclosure may refer to an artificial intelligence model including an artificial neural network, and may be trained by deep learning. For example, the neural network model 1000 may include at least one artificial neural network from a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), and a generative adversarial network (GAN). In addition, the neural network model 1000 may be of the same type as, for example but not limited to, an object recognition model, an automatic speech recognition model, a speech synthesis model, etc., but the neural network model 1000 according to the present disclosure is not limited to the above examples. The neural network model 1000 may not only be included in the electronic device 100 and implemented in the form of an on-device, but may also be included in an external device coupled to the electronic device 100.

[0047] refer to Figure 3, the neural network model 1000 may include a feature extraction module 1100 and a classification module 1200. The 'feature extraction module 1100' herein may refer to a module that extracts features useful to the corresponding neural network model 1000 from the input data and is capable of quantizing the features to output feature values. For example, the feature extraction module 1100 may include a convolution layer that performs convolution processing on each channel of the input data matrix to obtain a feature map. The feature values ​​output by the feature extraction module 1100 may be represented as a matrix / vector corresponding to the matrix / vector of the input data.

[0048] The 'classification module 1200' may refer to a module capable of obtaining output data corresponding to input data by identifying a category corresponding to the input data from a plurality of categories (or classifications) based on a feature value output by the feature extraction module 1100. The classification module 1200 may include a softmax layer, which may be configured to output a likelihood value for each of the plurality of categories by normalizing the output value.

[0049] In the electronic device 100, the neural network model 1000 as described above may be stored, and Figure 3 As shown, modules such as a symbol value mapping module 10, a merging module, a temporary answer acquiring module 30, a rule checking module 40, and a training determining module 50 may be stored therein.

[0050] The following will refer to Figure 1 and Figure 2 Description by Figure 3 Each module in the modules shown in FIG. 1 performs a specific process of the control method. However, it should be noted that Figure 3 The architecture of the software modules shown is only one embodiment, and in addition to the modules shown, new configurations may be included, or some modules may be omitted. In addition, at least two or more modules among the modules according to the present disclosure may be implemented as one module of an integrated type.

[0051] refer to Figure 1 and Figure 2 Based on the input data being input to the neural network model 1000, in operation S110, the electronic device 100 may obtain a first feature value corresponding to the input data through a feature extraction module 1100 included in the neural network model 1000. The feature value output by the feature extraction module 1100 of the neural network model 1000 may be referred to as a 'first feature value'.

[0052] As described above, the neural network model 1000 can obtain a feature value corresponding to the input data through the feature extraction module 1100, and obtain output data corresponding to the input data by inputting the obtained feature value to the classification module 1200. However, the electronic device 100 may not immediately input the first feature value obtained through the feature extraction module 1100 to the classification module 1200, and may be configured so that the first feature value can be input to the classification module 1200 through the symbol value mapping module 10 and the merging module.

[0053] In operation S120, the electronic device 100 may input the obtained first eigenvalue to the symbol value mapping module 10, and map information included in the first eigenvalue to symbol values ​​corresponding to the logically certain information and the logically uncertain information, respectively.

[0054] The 'symbol value mapping module 10' may refer to a module capable of mapping the first feature value obtained by the feature extraction module 1100 of the neural network model 1000 to a symbol value corresponding thereto. The symbol value mapping module 10 may be a module for converting continuous first feature values ​​into discrete symbol values.

[0055] The 'symbol value (or logic value)' about the predefined logic rule may include a symbol value corresponding to the logically determined information included in the first characteristic value, and a symbol value corresponding to the logically uncertain information. More specifically, the symbol value corresponding to the logically determined information may include a symbol value corresponding to the logically true information included in the first characteristic value and a symbol value corresponding to the logically false information.

[0056] The symbol value mapping module 10 can output a symbol value corresponding to each of the multiple parts according to which part of the multiple preset parts each of the information included in the first eigenvalue is included. For example, the symbol value may include values ​​1, -1, and 0, which are symbol values ​​corresponding to whether the information included in the first eigenvalue is logically true or not, whether the information is false or not, and whether the authenticity of the information is uncertain. If the information included in the first eigenvalue is a value corresponding to the true first part, the symbol value mapping module 10 can output 1 as a symbol value corresponding to the first part, if the information included in the first eigenvalue is a value corresponding to the false second part, then output -1 as a symbol value corresponding to the second part, and if the information included in the first eigenvalue is a value corresponding to the third part of uncertainty, then output 0 as a symbol value corresponding to the third part. If the information included in the first eigenvalue is uncertain information, the corresponding third part can be called a 'threshold range'.

[0057] According to the embodiments of the present disclosure, the symbol values ​​including values ​​corresponding to true, false or uncertain values ​​have been mainly described, but the embodiments are not limited thereto. That is, in order to solve the problem that there is no logical false situation, the symbol value may not include a value corresponding to false, and the value corresponding to uncertainty may be divided into multiple values ​​according to the degree of uncertainty.

[0058] In operation S130, the electronic device 100 may adjust the gradient corresponding to the uncertain information through the symbol value mapping module 10 so that the update of the uncertain information becomes possible. The neural network model 1000 may be trained by performing a back propagation process based on the gradient corresponding to the input data. However, when the symbol value mapping module 10 is operated to map the first eigenvalue to a symbol value corresponding to a constant as described above, since the gradient becomes 0 in all parts, the neural network model 1000 may not be trained by using the gradient.

[0059] If the information included in the first feature value corresponds to logically certain information, it may not be necessary to update the certain information, and if the information included in the first feature value corresponds to logically uncertain information, it may be necessary to train the uncertain information in the direction of matching the logical rule. Figure 2 As shown, the electronic device 100 according to the present disclosure can present directionality for training the neural network model 1000 by adjusting the gradient corresponding to the information only when the information included in the first eigenvalue is logically uncertain information.

[0060] like Figure 2 As shown, if in operation S130-Y, the information included in the first eigenvalue is not included in the threshold range corresponding to the logically uncertain information, then in operation S131, the electronic device 100 may maintain the gradient corresponding to the determined information as 0. On the other hand, if in operation S130-N, the information included in the first eigenvalue is included in the threshold range corresponding to the logically uncertain information, then in operation S132, the electronic device 100 may adjust the gradient corresponding to the uncertain information to a value other than 0. That is, the electronic device 100 may adjust the gradient corresponding to the uncertain information in the information included in the first eigenvalue by performing the processes of S130 to S132 for each of the information included in the first eigenvalue. Reference will be made to Figure 4 The process of adjusting the gradient corresponding to the first eigenvalue by the symbol value mapping module 10 is described in more detail.

[0061] When the symbol value corresponding to the first eigenvalue is obtained, in operation S140, the electronic device 100 may obtain a second eigenvalue in which the first eigenvalue and the symbol value are merged by inputting the first eigenvalue and the symbol value to a merging module.

[0062] The 'merging module' may refer to a module capable of merging the first feature value obtained by the feature extraction module 1100 of the neural network model 1000 and the symbol value obtained by the symbol value mapping module 10, and re-adjusting it into a feature value in a form that can be input to the classification module 1200 of the neural network model 1000. The term 'second feature value' may be used to distinguish the feature value output by the merging module from the first feature value output by the feature extraction module 1100 of the neural network model 1000.

[0063] 'Merge' may simply refer to a calculation of adding the first eigenvalue obtained by the feature extraction module 1100 of the neural network model 1000 and the symbol value obtained by the symbol value mapping module 10. The merging module may also perform a calculation of multiplying the first eigenvalue before adding the first eigenvalue to the symbol value and at least one predetermined weight value in the symbol value. For example, the merging module may obtain the second eigenvalue by multiplying a large weight value to create a greater influence of the symbol value relative to the second eigenvalue, and adding the first eigenvalue to the symbol value multiplied by the weight value. In addition to the above, after adding the first eigenvalue and the symbol value, the merging module may also perform a normalization process on values ​​between 0 and 1 by using a softmax function.

[0064] Based on obtaining the second eigenvalue as described above, the electronic device 100 is able to obtain the output data of the neural network model 1000 based on the second eigenvalue, wherein the symbol value is reflected on the first eigenvalue, and the output data of the neural network model 1000 is not obtained based on the first eigenvalue. However, whether to obtain the output data based on the second eigenvalue or to train the neural network model to obtain the output data after obtaining the updated second eigenvalue can be determined according to the following description and its process.

[0065] When the first eigenvalue and the symbol value corresponding to the first eigenvalue are obtained, the electronic device 100 may input the first eigenvalue and the symbol value to the provisional answer acquisition module 30 to obtain provisional answer information including certain information and uncertain information in operation S150.

[0066] The 'provisional answer acquisition module 30' may refer to a module capable of identifying definite information and uncertain information from the information included in the first eigenvalue, and outputting provisional answer information including the definite information and the uncertain information. The provisional answer acquisition module 30 may classify the information included in the first eigenvalue into logically definite information and logically uncertain information based on the symbol value corresponding to the first eigenvalue. The provisional answer acquisition module 30 may then analyze the meaning of the information included in the first eigenvalue, and obtain provisional answer information including the meaning of the definite information and the meaning of the uncertain information included in the first eigenvalue. The provisional answer acquisition module may obtain provisional answer information including information in which the uncertain information included in the first eigenvalue is randomly adjusted to an arbitrary value, as well as the definite information included in the first eigenvalue.

[0067] When the temporary answer information is obtained by the temporary answer acquisition module 30, the electronic device 100 can identify whether the meaning of the temporary answer information matches the predefined logical rule. When the temporary answer information corresponding to the first feature value is obtained, in operation S160, the electronic device 100 can input the temporary answer information to the rule checking module 40 and obtain score information related to the degree to which the temporary answer information matches the predefined logical rule.

[0068] The 'rule checking module 40' may refer to a module capable of outputting score information related to the degree to which the provisional answer information corresponding to the first feature value matches the predefined logical rule. The score information may be in the form of a score, wherein the more the provisional answer information matches the predefined logical rule, the higher its value is, and the less the provisional answer information matches the predefined logical rule, the lower its value is. In addition, the score information may be in the form of a rule loss, wherein the more the provisional answer information matches the predefined logical rule, the lower the value of the rule loss is, and the less the provisional answer information matches the predefined logical rule, the higher the value of the rule loss is.

[0069] Specifically, the rule checking module 40 can identify at least one logic rule applied to the input temporary answer information from a plurality of predefined logic rules, and output score information related to the degree of matching the at least one logic rule in which the input temporary answer information is identified. The predefined logic rules can be updated based on what is received from the external device.

[0070] Based on the score information obtained as described above, the electronic device 100 may determine, through the training determination module 50, whether to train the neural network model 1000 or obtain output data with respect to the input data.

[0071] The 'training determination module 50' may refer to a module that determines whether to further train the neural network model 1000 at a corresponding time point or provide output data to the user. Specifically, the training determination module 50 may determine whether to train the neural network model 1000 based on the second feature value received from the merging module and the score information received from the rule checking module 40.

[0072] Because the received second feature value includes information about the symbol value, the training determination module 50 can identify whether the first feature value includes uncertainty information based on the second feature value, and thus determine whether training of the neural network model 1000 is necessary to update the uncertainty information included in the first feature value. In addition, because the score information relates to the degree to which the symbol value corresponding to the first feature value matches the predefined logic rule, the training determination module 50 can determine whether training of the neural network model 1000 is necessary based on the received score information.

[0073] If the score information is less than the preset threshold value in operation S170-Y, the electronic device 100 may train the neural network model 1000 based on the adjusted gradient in operation S180. If the information included in the first feature value does not match the logic rule because it is greater than or equal to the preset threshold value, the electronic device 100 may also train the neural network model 1000 by using the adjusted gradient to update the uncertainty information included in the first feature value.

[0074] The training of the neural network model 1000 based on the score information refers to updating the parameters (weight values) of the neural network model 1000 in the direction where the score information is greater than or equal to a preset threshold value, thereby updating the symbol values ​​corresponding to the first eigenvalue, the temporary answer information, and the second eigenvalue, to update the first eigenvalue to include logically determined information instead of the uncertain information included in the first eigenvalue. The specific training process of the neural network model can be performed by using methods ranging from gradient descent to loss function reduction, as in the training process of the neural network model of the related art.

[0075] On the other hand, if the -N score information is a preset threshold or more in operation S170, the electronic device 100 may input the obtained second feature value to the classification module 1200 included in the neural network model 1000 in operation S190, and obtain output data about the input data. If the information included in the first feature value matches the logic rule by a preset threshold or more, the electronic device 100 may no longer train the neural network model 1000, and obtain output data based on the second feature value to provide to the user. The term 'output data' may refer to data that is ultimately provided to the user, and may be replaced by the term 'real answer information' to distinguish it from temporary answer information.

[0076] As described above, when the score information becomes a preset threshold or greater, since the uncertainty information is updated according to the neural network model 1000 trained based on the adjusted gradient, the electronic device 100 can obtain output data by inputting the updated second feature value into the classification module 1200 via the trained neural network model 1000, and the obtained output data can be provided to the user.

[0077] According to the embodiments of the present disclosure as described above, the neural network model 1000 can provide a result matching the logical rule regarding the question of whether the logical reasoning is appropriate. Therefore, the reliability of the neural network model 1000 can be improved, and the scope of use of the neural network model 1000 can also be expanded.

[0078] Because the existing neural network model 1000 solves problems such as those associated with safety according to a learning pattern based on learning data, it is difficult to solve the problem of forcibly applying logical rules regardless of the learning data. In addition, a large amount of learning data and a long learning time may be required to train the existing neural network model 1000, and if the distribution of the learning data is biased, there is a problem of outputting illogical results or even unethical results that violate social norms according to the distribution.

[0079] However, the neural network model 1000 according to the embodiment of the present disclosure can be trained to follow specific logical rules regardless of the amount of learning data, learning time, and distribution of learning data, and can solve exceptional cases where learning data based on logical rules cannot solve the problem. Then, because the neural network model 1000 according to the present disclosure can learn to distinguish between logically certain information and logically uncertain information, unnecessary processing can be minimized, and the efficiency and speed of the process of training the neural network model 1000 can be improved.

[0080] By minimizing the structural changes of the existing neural network model 1000, the module for implementing the control method according to the embodiment of the present disclosure can be implemented in the form of a software module that can be embedded in the existing neural network model 1000, or in the form of an independent neural network model 1000, and thus the addition and change of the logic rules applied to the neural network model 1000 can also be easily performed. The module according to the present disclosure may be a pluggable module received from an external device or server and coupled to an existing application or neural network model in the form of a plug-in, for example.

[0081] When the predefined logic rules are applied to the neural network model 1000 to solve problems with answers (e.g., solving puzzles such as Sudoku puzzles, Einstein puzzles, etc.), the above reference can be effectively utilized. Figures 1 to 3 Described embodiments.

[0082] Figure 4 is a diagram illustrating the operation of a symbol value mapping module according to an embodiment of the present disclosure in more detail.

[0083] refer to Figure 4 The symbol value mapping module 10 according to the present invention may specifically include a feature conversion module 11 and a gradient adjustment module 12.

[0084] The 'feature conversion module 11' may refer to a module capable of outputting a symbol value corresponding to a feature value by adjusting an input feature value. As described above, the feature conversion module 11 may perform a process of mapping information included in the first feature value to a symbol value corresponding to each of the logically determined information and the logically uncertain information (operation S120).

[0085] The 'gradient adjustment module 12' may refer to a module capable of adjusting a gradient corresponding to logically uncertain information in the information included in the first eigenvalue. As described above, the gradient adjustment module 12 may perform a process of adjusting a gradient only for uncertain information included in the first eigenvalue (operation S130).

[0086] The feature conversion module 11 and the gradient adjustment module 12 included in the symbol value mapping module 10 are only used to functionally divide and describe the operations performed according to the symbol value mapping module 10, and the operations according to each of the feature conversion module 11 and the gradient adjustment module 12 are not performed according to separate algorithms independent of each other, and can be determined by the function defined in Equation 2 as described below. Therefore, each process performed by the feature conversion module 11 and the gradient fine-tuning module 12 performed in the module referred to as the symbol value mapping module 10 will be described below.

[0087] Specifically, the symbol value mapping module 10 can be based on the function f(x) defined as the following equation 1 (hereinafter referred to as the 'discretization function'), when the first eigenvalue x is greater than or equal to 0.5, the output is 1 corresponding to the true symbol value, when the first eigenvalue x is less than -0.5, the output is -1 corresponding to the false symbol value, and when the first eigenvalue x is greater than or equal to -0.5 and less than 0.5, the output is 0 corresponding to the symbol value of uncertainty.

[0088]

[0089] That is, if the information included in the first eigenvalue x is a value within a preset threshold range corresponding to the uncertainty (i.e., if the information included in the first eigenvalue x is uncertain), the symbol value mapping module 10 can output 0 as the symbol value, and if the information included in the first eigenvalue x is a value outside the preset threshold range corresponding to the uncertainty (i.e., if the information included in the first eigenvalue x is certain), output 1 or -1 as the symbol value.

[0090] As described above, even if the continuous first eigenvalue is mapped to a discrete symbol value by the symbol value mapping module 10, if the feature conversion module 11 operates based on a function in which the gradient becomes 0 in all parts as in Equation 1, it may be difficult to train the neural network model 1000 by using the gradient. If the information included in the first eigenvalue corresponds to logically certain information, the certain information does not need to be updated, and if the information included in the first eigenvalue corresponds to logically uncertain information, the uncertain information needs to be learned in the direction of matching the logical rule.

[0091] Therefore, as long as the information included in the first eigenvalue is logically uncertain information, the symbol value mapping module 10 can adjust the gradient corresponding to the information. The symbol value mapping module 10 can adjust the gradient corresponding to the uncertain information in the information included in the first eigenvalue by performing the processes of S130 to S132 as described above for each of the information included in the first eigenvalue. The process of adjusting the gradient corresponding to the first eigenvalue by the symbol value mapping module 10 will be described in detail with reference to the following equations 2 to 7.

[0092] The symbol value mapping module 10 according to the present disclosure may be based on the function defined in the following equation 2: (hereinafter referred to as 'symbol value mapping function') rather than the discretization function f(x) defined in Equation 1. Specifically, the symbol value mapping function It can be defined as a function where the value of the gradient adjustment function s(x) multiplied by the gradient shaping function g(x) is added to the discretization function f(x) of Equation 1.

[0093]

[0094] The 'gradient shaping function g(x)' may be defined as the following equation 3. The gradient shaping function g(x) may be defined as a function that outputs 1 as a function value of information having a value greater than or equal to -0.5 and less than 0.5 in the information included in the first eigenvalue x, and outputs 0 as a function value of information having a value less than -0.5 or greater than or equal to 0.5 in the information included in the first eigenvalue x. If the information included in the first eigenvalue x is a value within a preset threshold range corresponding to uncertainty (i.e., if the information included in the first eigenvalue x is uncertain), 1 is output as a function value, and if the information included in the first eigenvalue x is a value outside the preset threshold range corresponding to uncertainty (i.e., if the information included in the first eigenvalue x is certain), 0 is output as a function value. The gradient shaping function g(x) may be reflected to the symbol value mapping function The gradient of , as described below with reference to Equations 6 and 7.

[0095]

[0096] The 'gradient adjustment function s(x)' may be a function in the form of a maximum integer function (floor function). However, the gradient adjustment function s(x) is not limited thereto and may be defined as various functions in addition to the form of an operation function. If the gradient adjustment function s(x) is defined as the maximum integer function of Equation 4, based on its definition, the function value of the gradient adjustment function s(x) may always be less than 1 / K, and the gradient of the gradient adjustment function s(x) may always be 1 in the differentiable part.

[0097]

[0098] As mentioned above, because the function value of the gradient adjustment function s(x) is always less than 1 / K, when K is large enough, the function value of the gradient adjustment function s(x) is close to 0, so the symbol value mapping function The function value of can be approximated to the function value of the discretized function f(x), as shown in the following equation 5. If the value of K is set to a sufficiently large value, the symbol value mapping function A function value (ie, a symbol value) approximating the discretized function f(x) may be output.

[0099]

[0100] The derivative f'(x) of the symbol value mapping function may be as shown in the following equation 6. As described above, the gradient of the gradient adjustment function may always be 1 in the differentiable part, and if K is large enough, the function value of the gradient adjustment function s(x) may be close to 0. Therefore, as shown in the following equation 7, the derivative of the symbol value mapping function It can approximate the function value of the gradient shaping function g(x). If the value of K is set to a sufficiently large value, the symbol value mapping function It may have a gradient that reflects the characteristics of the gradient shaping function f(x).

[0101] f′(x)=s′(x)g(x)+s(x)g′(x) Equation 6

[0102] ff′(x)≈g(x) Equation 7

[0103] However, as described above, if the information included in the first eigenvalue x is a value within the preset threshold range corresponding to the uncertainty, the gradient shaping function g(x) may output 1 as the function value, and if the information included in the first eigenvalue x is a value outside the preset threshold range corresponding to the uncertainty, 0 is output as the function value. Therefore, if the first eigenvalue x is a value outside the preset threshold range corresponding to the uncertainty, the symbol value mapping function The gradient of may be 0, and may be 1 if the first eigenvalue x is a value within a preset threshold range corresponding to the uncertainty.

[0104] Therefore, when the symbol value mapping module 10 is based on the symbol value mapping function as described above When in operation, the symbol value mapping module 10 can output discrete symbol values ​​as when operating based on the discretization function f(x), while making it possible to train the neural network model 1000 using a gradient only when the information included in the first eigenvalue x has a value within a preset threshold range corresponding to the uncertainty.

[0105] As described above, it may be necessary to set the value of K to a large value to achieve a degree to which the function value of the gradient adjustment function s(x) is close to 0. For example, a K value exceeding 105 may be preferred. The upper limit of the K value may be determined within a range in which overflow does not occur and meaningful processing is possible. For example, the value of K may be less than 10305. However, the K value according to the present disclosure is not limited to the above range.

[0106] Figure 5 is a diagram showing an embodiment related to the present disclosure applied to solve a problem of selecting an operation under a given condition according to an embodiment of the present disclosure.

[0107] As above reference Figures 1 to 4 As described, when the predefined logic rules are applied to the neural network model 1000 to solve a problem with a correct answer, the symbol value mapping module 10 can operate in a method of mapping at least one symbol value corresponding to the first eigenvalue, but when applied to solve the problem of selecting an operation under given conditions, multiple symbol values ​​need to be mapped to symbol values ​​corresponding to the first eigenvalue.

[0108] refer to Figure 5 According to the present disclosure, the symbol value mapping module 10 may include a conditional symbol value mapping module 13 and an action symbol value mapping module 14. The 'conditional symbol value mapping module 13' may refer to a module that maps a first feature value to a conditional symbol value, which is a symbol value corresponding to a predefined condition. The 'action symbol value mapping module 14' may refer to a module that maps a first feature value to an action symbol value, which is a symbol value corresponding to a predefined action.

[0109] When the first eigenvalue is input to the symbol value mapping module 10, the condition symbol value mapping module 13 may map the information included in the first eigenvalue to a different symbol value corresponding to each of the definite information and the uncertain information related to the predefined condition. The action symbol value mapping module 14 may map the information included in the first eigenvalue to a different symbol value corresponding to each of the definite information and the uncertain information related to the predefined condition.

[0110] For example, when the neural network model 1000 according to the present disclosure is a neural network model 1000 for determining the operation of an autonomous vehicle, if input data is input to the neural network model 1000 and a first feature value corresponding to the input data is obtained, the electronic device 100 may input the first feature value to the symbol value mapping module 10. The electronic device 100 may obtain a symbol value corresponding to each of the certain information and uncertain information related to the condition in which the possibility of a traffic accident is high from the information included in the first feature value through the conditional symbol value mapping module 13, and may obtain a symbol value corresponding to each of the certain information and uncertain information related to an action that can be taken when the possibility of a traffic accident is high through the action symbol value mapping module 14.

[0111] As described above, the electronic device 100 can adjust the gradient corresponding to the uncertain information through the symbol value mapping module 10 to make it possible to update the uncertain information, and can also obtain the second eigenvalue in which the first eigenvalue and the symbol value are merged by inputting the first eigenvalue and the symbol value in the merging module. Figures 1 to 4 The gradient adjustment process and the second feature acquisition process have been described, so a redundant description thereof will be omitted.

[0112] When the condition symbol value and the action symbol value correspond to the first characteristic value obtained, the electronic device 100 can input the first characteristic value, the condition symbol value and the action symbol value into the temporary answer acquisition module 30 to obtain temporary answer information including information about the condition included in the first characteristic value and information about the action corresponding thereto. The information about the condition included in the first characteristic value may include uncertain information (e.g., information about traffic conditions in front of and behind the autonomous driving car, etc.) and certain information (e.g., information that the current position of the autonomous driving car is in a school area, information about the speed limit of the current route of the autonomous driving car, etc.).

[0113] In the above example, the provisional answer information obtained through the provisional answer information may include information that the front pedestrian is within the preset risk distance as determined information related to the condition of high possibility of traffic accident occurrence, and information that the rear vehicle is within the preset risk distance as uncertain information related to the condition of high possibility of traffic accident occurrence. In addition, the provisional answer information may include information that it is determined to take the action of 'providing a warning notification' under the above conditions and information that it is uncertain to take the action of 'reducing the vehicle speed'.

[0114] When obtaining the provisional answer information corresponding to the first feature value, the electronic device 100 may input the provisional answer information to the rule checking module 40 to obtain score information related to the degree to which the provisional answer information matches the predefined logical rule.

[0115] In the above example, when the pedestrian in front is certain to be within the dangerous distance and the vehicle behind is uncertain to be within the dangerous distance, the electronic device 100 can obtain score information based on whether the actions of taking 'providing a warning notification' and 'reducing the vehicle speed' match the predefined logic rules. For example, the predefined logic rule can be a rule of "when the pedestrian is within the dangerous distance, perform an unconditional sudden stop". In this case, when the pedestrian in front is certain to be within the dangerous distance and the vehicle behind is uncertain to be within the dangerous distance, the electronic device 100 can obtain score information 0.7 as score information about taking the actions of 'providing a warning notification' and 'reducing the vehicle speed'.

[0116] When the score information about the provisional answer information is obtained, the electronic device 100 may determine whether to train the neural network model 1000 or obtain output data about the input data through the training determination module 50. If the score information is less than a preset threshold, the electronic device 100 may train the neural network model 1000 based on the adjusted gradient. On the other hand, if the score information is greater than or equal to the preset threshold, the electronic device 100 may input the obtained second feature value to the classification module 1200 included in the neural network model 1000 to obtain output data about the input data.

[0117] In the above example, if the threshold of the score information is preset to 0.08, since the score information will be less than the preset threshold, the electronic device 100 can train the neural network model 1000 based on the adjusted gradient. Therefore, when the pedestrian in front is certain to be within the dangerous distance and the vehicle behind is uncertain to be within the dangerous distance, the electronic device 100 can obtain temporary response information including information that it is certain to take the action of 'suddenly stopping the vehicle', rather than information that it is uncertain to take the action of 'reducing the vehicle speed'.

[0118] According to the above reference Figure 5 In the described embodiment, regarding the problem of selecting an operation under a given condition, the electronic device 100 can provide output data matching the logic rule by updating the uncertainty information to match the predefined logic rule.

[0119] As in the above-mentioned example, by applying rules or general rules to problems that occur with exceptions that cannot be solved only by statistical determination based on learning data, the safety issues pointed out with respect to the neural network model 1000 can be solved. In addition, with respect to the problem that it is difficult to obtain a large amount of learning data in various actual situations, since it can be applied to the neural network model 1000 to match the logical rules, the neural network model 1000 can be stably operated even when the learning data is insufficient or when there is no learning data.

[0120] Figure 6 is a schematic diagram showing an embodiment of training a neural network model based on score information related to a degree of matching learning data according to an embodiment of the present disclosure.

[0121] As described above, by training the neural network model 1000 based on the score information related to the degree to which the temporary answer information matches the predefined logical rules, the electronic device 100 can provide output data of the neural network model 1000 matching the logical rules. Hereinafter, the score information related to the degree to which the temporary answer information matches the predefined logical rules can be referred to as 'first score information'.

[0122] However, by only training the neural network model 1000 to match the logical rules, refer to Figure 5 Information that taking the action of 'reducing vehicle speed' is uncertain in the described example may be updated only to information that taking the action of 'suddenly stopping vehicle' is certain, and information that the rear vehicle is within a dangerous distance is uncertain may not be updated to certain information.

[0123] Regarding unreasonable problems to which logical rules cannot be applied or solved according to logical rules, it may be preferred to provide a result with a high statistical probability rather than a result based on logical rules. Therefore, the electronic device 100 may train the neural network model 1000 based on the score information related to the degree to which the output data matches the learning data of the neural network model 1000 and the score information related to the degree to which the predefined logical rules are matched. Hereinafter, the score information related to the degree to which the output data matches the learning data of the neural network model 1000 may be distinguished from the first score information and referred to as 'second score information'.

[0124] Specifically, refer to Figure 6 , the electronic device 100 may input the provisional answer information to the rule checking module 40 to obtain not only the first score information related to the degree to which the provisional answer matches the logic rule, but also the second score information related to the degree to which the output data obtained by the classification module 1200 of the neural network model 1000 matches the learning data of the neural network model 1000. The second score information may include information on a loss function, which is one of the indicators used in the training of the neural network model 1000 of the related art. Then, the training determination module 50 may not only train the neural network model 1000 to match the predefined logic rule based on the first score information, but also train the neural network model 1000 to match the previously constructed learning data based on the second score information.

[0125] The electronic device 100 may train the neural network model 1000 based on the comprehensive score information in which the first score information and the second score information are combined. The comprehensive score information may be obtained not only by adding the first score information and the second score information, but also by multiplying a predetermined weight value by at least one of the first score information and the second score information and then adding them. For example, with respect to a problem in which a logical rule must be clearly followed, a higher weight value may be applied to the first score information rather than the second score information.

[0126] According to the above reference Figure 6 In the described embodiment, in addition to applying logical rules to the neural network model 1000, the electronic device 100 can also train the neural network model 1000 to match learning data about unreasonable problems that cannot be applied with logical rules or that are solved according to logical rules, so that the neural network model 1000 outputs a result that is statistically highly likely.

[0127] Figure 7 is a diagram illustrating an embodiment of obtaining temporary answer information through a classification module of a neural network model according to an embodiment of the present disclosure.

[0128] refer to Figure 7, an embodiment in which the provisional answer information corresponding to the first feature value is obtained by the provisional answer acquisition module 30 included in the electronic device 100 has been described, but according to another embodiment of the present disclosure, the classification module 1200 may be implemented to output not only the output data as the real answer provided to the user, but also the provisional answer information according to the present disclosure. In other words, the functions that can be performed by the provisional answer acquisition module 30 may be implemented by the classification module 1200 included in the neural network model 1000.

[0129] The first feature value obtained by the feature extraction module 1100 of the neural network model 1000 can be input to the classification module 1200, and the symbol value obtained by the symbol value mapping module 10 can bypass the merging module and the training determination module 50, or can be input to the classification module 1200 in the form of including the second feature value.

[0130] The classification module 1200 may then classify the information included in the first eigenvalue into logically determined information and logically uncertain information based on the symbol value corresponding to the first eigenvalue. In addition, the classification module 1200 may analyze the meaning of the information included in the first eigenvalue to obtain temporary answer information, which includes the meaning of the determined information and the meaning of the uncertain information included in the first eigenvalue. The classification module 1200 may obtain the temporary answer information based on which category the information included in the first eigenvalue corresponds to among the predefined multiple categories.

[0131] When the provisional answer information is obtained through the classification module 1200, the electronic device 100 may input the provisional answer information to the rule checking module 40 to obtain score information related to whether the provisional answer matches the predefined logic rule, and the training determination module 50 may determine the training of the neural network model 1000 based on the score information. When the training of the neural network model 1000 is sufficiently performed to the extent that the score information is greater than or equal to a preset threshold, the electronic device 100 may input the second feature updated according to the training of the neural network model 1000 to the classification module 1200 and obtain output data provided to the user.

[0132] refer to Figure 7 , since the present disclosure can be applied by obtaining temporary answer information by using the structure of the existing neural network model 1000 without a separate module for obtaining temporary answer information, multiple modules according to the present disclosure and the existing neural network model 1000 can be more effectively combined.

[0133] Figure 8 is a block diagram briefly showing a hardware configuration of an electronic device according to an embodiment of the present disclosure, and Fig. 9 is a block diagram illustrating in detail a hardware configuration of an electronic device according to an embodiment of the present disclosure.

[0134] refer to Figure 8 , the electronic device 100 according to an embodiment of the present disclosure may include a memory 110 and a processor 120 .

[0135] refer to Fig. 9 , the electronic device 100 according to the embodiment of the present disclosure may further include a communicator 130, a sensor 140, an inputter 150, and an outputter 160. However, Figure 8 and Fig. 9 The configuration shown is only an example and may be Figure 8 and Fig. 9 Add new configurations as shown or you can add new configurations from Figure 8 and Fig. 9 A part of the configuration is omitted in the configuration shown.

[0136] The memory 110 may store at least one instruction related to the electronic device 100. The memory 110 may store an operating system (O / S) for operating the electronic device 100. In addition, various software programs or applications may be stored in the memory 110 for operating the electronic device 100 according to various embodiments. The memory 110 may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0137] The memory 110 may store various software modules for the electronic device 100 to operate according to various embodiments of the present disclosure, and the processor 120 may execute the various software modules stored in the memory 110 to control the operation of the electronic device 100. The memory 110 may be accessed by the processor 120, and data reading / recording / modification / update, etc. may be performed by the processor 120.

[0138] The term memory may be used to mean including the memory 110, a read-only memory (ROM, not shown) within the processor 120, a random access memory (RAM, not shown), or a memory card (not shown, such as a micro SD card, a memory stick) installed in the electronic device 100.

[0139] The neural network model 1000 according to an embodiment of the present disclosure may be stored in the memory 110, and may also store software modules such as the symbol value mapping module 10, the merging module, the temporary answer acquisition module 30, the rule checking module 40, and the training determination module 50. In addition, various data such as input data, first eigenvalues, symbol values, gradients, second eigenvalues, temporary answer information, score information, output data, etc. may be stored in the memory 110.

[0140] Besides, various necessary information within the scope of achieving the purpose of the present disclosure may be stored in the memory 110 , and the information stored in the memory 110 may be received from an external device or updated based on a user's input.

[0141] The processor 120 may control the overall operation of the electronic device 100. The processor 120 may be coupled to the configuration of the electronic device 100 including the memory 110, the communicator 130, the sensor 140, the inputter 150, and the outputter 160, and may control the overall operation of the electronic device 100 by executing at least one instruction stored in the memory 110 as described above.

[0142] The processor 120 may be implemented in various ways. For example, the processor 120 may be implemented as at least one of an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, a hardware control logic, a hardware finite state machine (FSM), or a digital signal processor (DSP). The term processor 120 in the present disclosure may be used to include a central processing unit (CPU), a graphics processing unit (GPU), a main processing unit (MPU), etc.

[0143] According to an embodiment of the present disclosure, the processor 120 can input the input data into the neural network model 1000, obtain a first eigenvalue corresponding to the input data through the feature extraction module 1100 included in the neural network model 1000, input the obtained first eigenvalue into the symbol value mapping module 10, map the information included in the first eigenvalue to a symbol value corresponding to each of the logically certain information and the logically uncertain information, adjust the gradient corresponding to the uncertain information through the symbol value mapping module 10 so that the update of the uncertain information becomes possible, input the first eigenvalue and the symbol value into the merging module to obtain a first eigenvalue and a symbol value corresponding to each of the logically certain information and the logically uncertain information. The first eigenvalue and the symbol value are combined into a second eigenvalue, the first eigenvalue and the symbol value are input to the temporary answer acquisition module 30 to obtain temporary answer information including certain information and uncertain information, the temporary answer information is input to the rule checking module 40 to obtain score information related to the degree to which the temporary answer information matches the predefined logical rule, if the score information is less than a preset threshold, the neural network model 1000 is trained based on the adjusted gradient through the training determination module 50, and if the score information is greater than or equal to the preset threshold, the obtained second eigenvalue is input to the classification module 1200 included in the classification module 1200 to obtain output data about the input data. As has been referred to above Figures 1 to 7 Various embodiments according to the present disclosure based on the control of the processor 120 are described, so a redundant description thereof may be omitted.

[0144] The communicator 130 including a circuit may communicate with an external device. The processor 120 may receive various data or information from an external device coupled through the communicator 130, and may transmit various data or information to the external device.

[0145] The communicator 130 may include at least one of a Wi-Fi module, a Bluetooth module, a wireless communication module, and a near field communication (NFC) module. Each of the Wi-Fi module and the Bluetooth module may communicate via a Wi-Fi method and a Bluetooth method. When using the Wi-Fi module or the Bluetooth module, various connection information such as a service set identifier (SSID) may be first sent and received, and various information may be sent and received after communication coupling using the connection information.

[0146] In addition, the wireless communication module may perform communication according to various communication standards such as IEEE, Zigbee, third generation (3G), third generation partnership project (3GPP), long term evolution (LTE), fifth generation (5G), etc. The NFC module may perform communication using a near field communication (NFC) method using a 13.56 MHz band among various radio frequency identification (RF-ID) bands such as, but not limited to, 135 kHz, 13.56 MHz, 433 MHz, 860-960 MHz, 2.45 GHz, etc.

[0147] According to various embodiments of the present disclosure, the processor 120 may receive information about predefined logic rules through the communicator 130. For example, because the predefined logic rules may vary according to regulations or rules based on countries or regions, the processor 120 may update the logic rules applied in the present disclosure by receiving information about the predefined logic rules through the communicator 130. In addition, the processor 120 may receive information related to the neural network model 1000, the symbol value mapping module 10, the merging module, the temporary answer acquisition module 30, the rule checking module 40, the training determination module 50, etc. according to the present disclosure through the communicator 130.

[0148] The outputter may include a circuit, and the processor 120 may output various functions that the electronic device 100 may perform through the outputter 160. The outputter 160 may include at least one of a display, a speaker, and an indicator.

[0149] The display may output image data under the control of the processor 120. The display may output an image pre-stored in the memory 110 under the control of the processor 120. The display according to an embodiment of the present disclosure may also display a user interface stored in the memory 110. The display may be implemented as a liquid crystal display (LCD) panel, an organic light emitting diode (OLED), etc. Depending on the situation, the display may also be implemented as a flexible display, a transparent display, etc. However, the display according to the present disclosure is not limited to a specific type.

[0150] The speaker may output audio data under the control of the processor 120 , and the indicator may be lit under the control of the processor 120 .

[0151] According to various embodiments of the present disclosure, the processor 120 may provide output data obtained based on the present disclosure to the user through the outputter 160. The processor 120 may visually provide the output data to the user through the display, and also provide the output data to the user in the form of a voice signal through the speaker.

[0152] The input 150 may include a circuit, and the processor 120 may receive a user command for controlling the operation of the electronic device 100 through the input 150. The input 150 may include a configuration such as a microphone, a camera (not shown), a remote control signal receiver (not shown), etc. The input 150 may also be implemented in the form of being included in a display as a touch screen.

[0153] According to various embodiments of the present disclosure, the processor 120 may receive a user command through the input 150 to start the operation of the processor 120 according to the present disclosure. Regardless of whether the training of the neural network model 1000 is sufficient, the processor 120 may receive a user command through the input 150 to provide output data corresponding to the input data through the current neural network model 1000.

[0154] The control method of the electronic device 100 according to the above-described embodiment may be implemented as a program and provided to the electronic device 100. The program including the control method of the electronic device 100 may be stored in a non-transitory computer-readable medium and provided therein.

[0155] With respect to a non-transitory computer-readable recording medium including a program for executing a control method of the electronic device 100, the control method of the electronic device 100 may include inputting input data into the neural network model 1000, obtaining a first feature value corresponding to the input data through a feature extraction module 1100 included in the neural network model 1000, inputting the obtained first feature value into a symbol value mapping module 10 included in the electronic device, mapping information included in the first feature value to a symbol value corresponding to each of logically certain information and logically uncertain information, adjusting a gradient corresponding to the uncertain information through the symbol value mapping module 10 so that updating of the uncertain information becomes possible, obtaining the first feature value and the symbol value by inputting the first feature value and the symbol value in a merging module included in the electronic device 100, a second eigenvalue in which the first eigenvalue and the symbol value are combined, obtaining temporary answer information including certain information and uncertain information by inputting the first eigenvalue and the symbol value into a temporary answer acquisition module 30 included in the electronic device 100, obtaining score information related to the degree to which the temporary answer information matches a predefined logical rule by inputting the temporary answer information into a rule checking module 40 included in the electronic device 100, and if the score information is less than a preset threshold, training the neural network model 1000 based on the adjusted gradient by a training determination module 50 included in the electronic device 100, and obtaining output data about the input data by inputting the obtained second eigenvalue into a classification module 1200 included in the neural network model 1000 when the score information is greater than or equal to the preset threshold.

[0156] The non-transitory computer-readable medium may refer to a medium that stores data semi-permanently rather than storing data for a very short time and can be read by the device, such as a register, a cache, or the memory 110. The aforementioned various applications or programs may be stored in a non-transitory computer-readable medium, such as, but not limited to, a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a universal serial bus (USB), a memory card, a read-only memory (ROM), etc., and may be provided.

[0157] The control method of the electronic device 100 and the computer-readable recording medium including a program for executing the control method of the electronic device 100 have been briefly described, but this is only to omit the redundant description thereof, and various embodiments regarding the electronic device 100 may also be applied to the control method of the electronic device 100 and the computer-readable recording medium including a program for executing the control method of the electronic device 100.

[0158] According to various embodiments of the present disclosure as described above, the usability and learning efficiency of the neural network model 1000 can be extended by applying logical rules while minimizing changes to the structure of the existing neural network model 1000, and security issues regarding the neural network model 1000 can be resolved by providing logically reasonable results in exceptional cases that cannot be resolved using only statistical determination based on learning data. In addition, according to the present disclosure, by classifying statistical information and logically specific information, unnecessary processing in the learning process of the neural network model 1000 can be minimized.

[0159] Because the existing neural network model 1000 solves problems such as those associated with safety according to a learning pattern based on learning data, there is a limitation that it is difficult to solve the problem that logical rules must be applied forcibly regardless of the learning data. In addition, in order to train the existing neural network model 1000, a large amount of learning data and a long learning time may be required, and if the distribution of the learning data is biased, there is a problem of outputting illogical results or even unethical results that violate social norms according to the distribution.

[0160] However, the neural network model 1000 can be trained to follow specific logical rules regardless of the amount of learning data, learning time, and distribution of learning data, and can solve exceptions that cannot be solved based on learning data according to the logical rules. Because the neural network model 1000 according to the present disclosure can learn to distinguish between logically certain information and logically uncertain information, unnecessary processing can be minimized, and the efficiency and speed of the process of training the neural network model 1000 can be improved.

[0161] In addition, by minimizing the structural changes of the existing neural network model 1000, the module for implementing the control method according to the present disclosure can be implemented in the form of an embedded software module or in the form of an independent neural network model 1000 in the existing neural network model 1000, and therefore the addition and change of logical rules applied to the neural network model 1000 can also be easily performed.

[0162] In addition to the above, by training the neural network model 1000 to match learning data about irrational problems to which logical rules cannot be applied or to which the problems can be solved according to logical rules, in addition to applying logical rules to the neural network model 1000, the neural network model 1000 can be made to output results that are statistically highly likely.

[0163] The functions related to the neural network model as described above can be executed by the memory and the processor.

[0164] The processor may include multiple processors. One or more processors may be general-purpose processors such as CPUs, APs, etc., graphics-specific processors such as GPUs, video processing units (VPUs), etc., or artificial intelligence-specific processors such as neural processing units (NPUs).

[0165] One or more processors can control processing input data according to predefined operating rules stored in non-volatile memory and volatile memory or artificial intelligence model. The predefined operating rules or artificial intelligence model can be characterized by learning generation.

[0166] Generating by learning refers to generating the desired features or predefined operating rules of the artificial intelligence model by applying the learning algorithm to multiple learning data. The learning can be performed by the machine itself in which the artificial intelligence is performed according to the embodiment or by a separate server / system.

[0167] The artificial intelligence model may be composed of multiple neural network layers. Each layer may have multiple weight values, and the processing of the layer may be performed by processing the processing result of the previous layer and the multiple weight values. Examples of neural networks may include, for example, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), and deep Q networks, unless otherwise stated, the neural networks of the present disclosure are not limited to the above examples.

[0168] A learning algorithm may be a method of using a plurality of learning data to train a predetermined subject machine (e.g., a robot) to determine or predict on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, and learning algorithms are not limited to the above examples unless otherwise specified.

[0169] The machine-readable storage medium may be provided in the form of a non-transitory storage medium. In this document, 'non-transitory' simply means that the storage medium is tangible, does not include signals, and the term does not distinguish between data stored semi-permanently in the storage medium and data stored temporarily. For example, a 'non-transitory storage medium' may include a buffer that temporarily stores data.

[0170] According to an embodiment of the present disclosure, a method according to one or more embodiments including a computer program product may be provided. The computer program product may be exchanged between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or distributed online through an application store (e.g., PLAYSTORE), or distributed directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be at least temporarily stored in a storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server, or temporarily generated.

[0171] Each element (e.g., module or program) according to various embodiments of the present disclosure may include a single entity or multiple entities, and some of the above-mentioned sub-elements may be omitted, or different sub-elements may be further included in various embodiments. Alternatively or additionally, some elements (e.g., modules or programs) may be integrated into one entity to perform the same or similar functions performed by each corresponding element before integration.

[0172] According to various embodiments, operations performed by a module, program or other element may be performed sequentially in parallel, repeatedly or in a heuristic manner, or at least some operations may be performed in a different order, omitted or different operations may be added.

[0173] The term "part" or "module" used in the present disclosure may include a unit configured as hardware, software or firmware, and may be used interchangeably with terms such as, but not limited to, logic, logic block, component, circuit, etc. A "part" or "module" may be a component formed as a whole, or a minimum unit or part of a component that performs one or more functions. For example, a module may be configured as an application specific integrated circuit (ASIC).

[0174] One or more embodiments of the present disclosure may be implemented with software including instructions stored in a machine-readable storage medium (e.g., a computer). The machine may call the instructions stored in the storage medium, and as a device capable of operating according to the called instructions, may include an electronic device according to the embodiment (e.g., electronic device 100).

[0175] Based on the instructions executed by the processor, the processor may use different elements to perform functions corresponding to the instructions directly or under the control of the processor. The instructions may include codes generated by a compiler or executed by an interpreter.

[0176] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. An electronic device configured to apply logical constraints to a neural network model used to determine operation of an autonomous vehicle, the electronic device comprising: A memory configured to store a neural network model, a symbol value mapping module, a merging module, a temporary answer acquisition module, a rule checking module, and a training determination module; and a processor configured to control the electronic device by executing at least one instruction stored in the memory, Wherein, the processor is configured to: Based on the input data being input into the neural network model, a first feature value corresponding to the input data is obtained by a feature extraction module included in the neural network model, inputting the first eigenvalue into the symbol value mapping module, mapping information included in the first feature value to a symbol value corresponding to each of logically certain information and uncertain information, wherein the logically certain information and uncertain information are related to actions that can be taken when there is a high possibility of a traffic accident occurring, The gradient corresponding to the uncertain information is adjusted by the symbol value mapping module so as to make it possible to update the uncertain information. By inputting the first eigenvalue and the symbol value into the merging module, obtaining a second eigenvalue in which the first eigenvalue and the symbol value are merged, By inputting the first feature value and the symbol value into the provisional answer acquisition module to obtain provisional answer information including the certain information and the uncertain information, obtaining score information related to the degree to which the provisional answer information matches a predefined logical rule by inputting the provisional answer information into the rule checking module, Based on the score information being less than a preset threshold, and based on the first score information related to the degree of matching the logic rule and the second score information related to the degree of matching the output data with the learning data of the neural network model, training the neural network model based on the adjusted gradient by the training determination module, Based on the score information being greater than or equal to the preset threshold, obtaining output data about the input data by inputting the obtained second feature value into a classification module included in the neural network model, and The operation of the autonomous driving vehicle is selected by updating the uncertain information to match the predefined logical rules and outputting output data matching the logical rules.

2. The electronic device according to claim 1, wherein: The processor is further configured to: training the neural network model based on the adjusted gradient so that the uncertainty information is updated, and Based on the uncertainty information being updated and the score information being greater than or equal to a preset threshold, the output data is obtained by inputting the second feature value updated by the trained neural network model into the classification module.

3. The electronic device according to claim 1, wherein: The processor is further configured to: The symbol value mapping module adjusts the gradient corresponding to the uncertain information to a value other than 0 so that the updating of the uncertain information becomes possible, and The gradient corresponding to the determination information is maintained at 0 so that no update of the determination information is performed.

4. The electronic device according to claim 1, in, The symbol value mapping module comprises: a conditional symbol value mapping module configured to map the first feature value to a conditional symbol value corresponding to a predefined condition, and an action symbol value mapping module, configured to map the first feature value to an action symbol value corresponding to a predefined action, and Wherein, the processor is further configured to: By the conditional symbol value mapping module, information included in the first feature value is mapped to a different conditional symbol value relative to each of the certain information and the uncertain information related to the condition, and By the action symbol value mapping module, information included in the first feature value is mapped to a different action symbol value relative to each of the certain information and the uncertain information related to the action.

5. The electronic device according to claim 4, wherein: The processor is further configured to: inputting the first feature value, the condition symbol value and the action symbol value into the provisional answer acquisition module, and Provisional answer information including information about a condition included in the first feature value and information about an action corresponding to the information about the condition included in the first feature value is obtained.

6. The electronic device according to claim 1, wherein: The symbol value corresponding to the determination information includes a symbol value corresponding to when the information included in the first characteristic value is logically true and a symbol value corresponding to when the information included in the first characteristic value is logically false.

7. A control method of an electronic device, the electronic device being configured to apply a logical constraint to a neural network model for determining operation of an autonomous vehicle, the method comprising: Based on input data input into the neural network model, obtaining a first feature value corresponding to the input data through a feature extraction module included in the neural network model; By inputting the first feature value into a symbol value mapping module included in the electronic device, information included in the first feature value is mapped to a symbol value corresponding to each of logically certain information and uncertain information, wherein the logically certain information and uncertain information are related to actions that can be taken when there is a high possibility of a traffic accident occurring; adjusting the gradient corresponding to the uncertain information by the symbol value mapping module so as to make it possible to update the uncertain information; By inputting the first eigenvalue and the symbol value into a merging module included in the electronic device, obtaining a second eigenvalue in which the first eigenvalue and the symbol value are merged; Obtaining provisional answer information including the certain information and the uncertain information by inputting the first characteristic value and the symbol value into a provisional answer acquisition module included in the electronic device; obtaining score information related to the degree to which the provisional answer information matches a predefined logic rule by inputting the provisional answer information into a rule checking module included in the electronic device; and Based on the score information being less than a preset threshold value, and based on the first score information related to the degree of matching the logic rule and the second score information related to the degree of matching the output data with the learning data of the neural network model, obtaining output data about the input data by training the neural network model based on the adjusted gradient through a training determination module included in the electronic device; Based on the score information being greater than or equal to the preset threshold, inputting the obtained second feature value into a classification module included in the neural network model; and The operation of the autonomous driving vehicle is selected by updating the uncertain information to match the predefined logical rules and outputting output data matching the logical rules.

8. The method according to claim 7, wherein: The control method of the electronic device further includes: training the neural network model based on the adjusted gradient so that the uncertainty information is updated; and Based on the uncertainty information being updated and the score information being greater than or equal to a preset threshold, the output data is obtained by inputting the second feature value updated by the trained neural network model into the classification module.

9. The method according to claim 7, wherein: Adjusting the gradient includes: adjusting the gradient corresponding to the uncertain information to a value other than 0 by the symbol value mapping module so as to make it possible to update the uncertain information, and The gradient corresponding to the determination information is maintained at 0 so that no update of the determination information is performed.

10. The method according to claim 7, in, The symbol value mapping module comprises: a conditional symbol value mapping module configured to map the first feature value to a conditional symbol value corresponding to a predefined condition, and an action symbol value mapping module, configured to map the first feature value to an action symbol value corresponding to a predefined action, and Wherein, mapping the symbol value comprises: By means of the conditional symbol value mapping module, information included in the first characteristic value is mapped to a different conditional symbol value relative to each of the certain information and the uncertain information related to the condition, and By the action symbol value mapping module, information included in the first feature value is mapped to a different action symbol value relative to each of the certain information and the uncertain information related to the action.

11. The method according to claim 10, wherein: Obtaining the temporary answer information includes: inputting the first feature value, the condition symbol value and the action symbol value into the provisional answer acquisition module, and Provisional answer information including information about a condition included in the first feature value and information about an action corresponding to the information about the condition included in the first feature value is obtained.

12. The method according to claim 7, wherein: The symbol value corresponding to the determination information includes: a symbol value corresponding to when the information included in the first eigenvalue is logically true, and Corresponding to a sign value when information included in the first eigenvalue is logically false.

13. A computer-readable recording medium comprising a program for executing a method for controlling an electronic device, the method for controlling an electronic device comprising: Based on input data input to a neural network model for determining the operation of the autonomous driving vehicle, a first feature value corresponding to the input data is obtained by a feature extraction module included in the neural network model; By inputting the first feature value into a symbol value mapping module included in the electronic device, information included in the first feature value is mapped to a symbol value corresponding to each of logically certain information and logically uncertain information, wherein the logically certain information and the uncertain information are related to actions that can be taken when there is a high possibility of a traffic accident occurring; adjusting the gradient corresponding to the uncertain information by the symbol value mapping module so as to make it possible to update the uncertain information; By inputting the first eigenvalue and the symbol value into a merging module included in the electronic device, obtaining a second eigenvalue in which the first eigenvalue and the symbol value are merged; Obtaining provisional answer information including the certain information and the uncertain information by inputting the first characteristic value and the symbol value into a provisional answer acquisition module included in the electronic device; obtaining score information related to the degree to which the provisional answer information matches a predefined logic rule by inputting the provisional answer information into a rule checking module included in the electronic device; and Based on the score information being less than a preset threshold value, and based on the first score information related to the degree of matching the logic rule and the second score information related to the degree of matching the output data with the learning data of the neural network model, obtaining output data about the input data by training the neural network model based on the adjusted gradient through a training determination module included in the electronic device; Based on the score information being greater than or equal to the preset threshold, inputting the obtained second feature value into a classification module included in the neural network model; and The operation of the autonomous driving vehicle is selected by updating the uncertain information to match the predefined logical rules and outputting output data matching the logical rules.

Images