Electronic device for detecting abnormality

Abstract

An electronic device for detecting anomalies includes a processor and a memory storing instructions to cause the processor to input a normal input image to a pre-learned autoencoder, receive a normal output image from the autoencoder as an output related to the normal input image, and output the normal input image to the autoencoder. On the basis of a normal input image and a normal output image recognition threshold value, an input image to be determined is input to the automatic encoder, an output image to be determined is received from the automatic encoder as an output related to the input image to be determined, the input image to be determined is divided into a plurality of input regions, and the input regions are input to the automatic encoder. A determination target output image is divided into a plurality of output regions to respectively correspond to a plurality of input regions, first reconstruction errors associated with each of the corresponding input regions and each of the output regions are identified, and the determination target output image is determined on the basis of a comparison result between a threshold value and each of the first reconstruction errors. An abnormal region in each corresponding input region and each corresponding output region is identified.

Description

Technical Field

[0001] The invention relates to an electronic device for detecting anomalies, in particular to an electronic device for detecting anomalies by using an automatic encoder. Background Art

[0002] The contents described in this section are only used to provide background information for this embodiment and do not constitute prior art.

[0003] Recently, the method of using an autoencoder to judge whether it is normal or abnormal has been widely used. In order to improve the performance of the autoencoder, the input and output of the autoencoder can be used to make the autoencoder that has been learned re-learn.

[0004] On the other hand, in order to improve the performance of an autoencoder without retraining an already learned autoencoder, a new approach involving the application of autoencoders is required. Summary of the invention

[0005] Problem that the invention aims to solve

[0006] The object of the present invention is to provide an electronic device for detecting abnormalities, that is, to use a pre-learned autoencoder to improve the accuracy and reliability of the judgment of normality made by the autoencoder.

[0007] Furthermore, an object of the present invention is to provide an electronic device for detecting anomalies, that is, capable of improving the performance of an autoencoder without relearning a pre-learned autoencoder.

[0008] The purpose of the present invention is not limited to the purpose mentioned above, and other purposes and advantages of the present invention not mentioned can be understood by the following description, and can be more clearly understood by the embodiments of the present invention. In addition, the purposes and advantages of the present invention can obviously be achieved by the methods and combinations thereof shown in the scope of the invention claims.

[0009] Means used to solve problems

[0010] An electronic device for detecting abnormalities according to an embodiment of the present invention comprises: a processor; and a memory, operatively connected to the processor, wherein the memory stores instructions for causing the processor to execute the following steps when the memory is running: inputting a normal input image into a pre-learned autoencoder, receiving a normal output image from the autoencoder as an output related to the normal input image, inputting a determination object input image into the autoencoder based on a recognition threshold of the normal input image and the normal output image, receiving a determination object output image from the autoencoder as an output related to the determination object input image, segmenting the determination object input image into a plurality of input regions, segmenting the determination object output image into a plurality of output regions to correspond to the plurality of input regions respectively, identifying a plurality of first reconstruction errors associated with each of the corresponding plurality of input regions and each of the corresponding plurality of output regions, and identifying an abnormal region in each of the corresponding plurality of input regions and each of the corresponding plurality of output regions based on respective comparison results between the threshold and the plurality of first reconstruction errors.

[0011] Furthermore, the above-mentioned instruction causes the above-mentioned processor to perform the following steps: identifying the first input area among the above-mentioned multiple input areas and the first output area among the above-mentioned multiple output areas corresponding to the above-mentioned first input area, identifying the above-mentioned first reconstruction judgment error included in the above-mentioned multiple first reconstruction errors as the first reconstruction judgment error related to the above-mentioned first input area and the above-mentioned first output area, and comparing the above-mentioned threshold value with the above-mentioned first reconstruction judgment error to identify whether the corresponding above-mentioned first input area and the above-mentioned first output area are normal.

[0012] Furthermore, the above-mentioned instructions cause the above-mentioned processor to perform the following steps: segmenting the above-mentioned normal input image into multiple normal input areas, segmenting the above-mentioned normal output image into multiple normal output areas to respectively correspond to the above-mentioned multiple normal input areas, identifying multiple second reconstruction errors associated with each of the corresponding multiple normal input areas and each of the corresponding multiple normal output areas, and identifying the above-mentioned threshold based on one of the above-mentioned multiple second reconstruction errors.

[0013] Furthermore, the above-mentioned instruction enables the above-mentioned processor to perform the following steps: calculating the above-mentioned multiple first reconstruction errors, the above-mentioned multiple second reconstruction errors and the above-mentioned threshold based on the structural similarity index measure (SSIM), and the above-mentioned threshold is set based on the minimum value of the above-mentioned multiple second reconstruction errors.

[0014] Furthermore, the above-mentioned instructions cause the above-mentioned processor to perform the following steps: identifying the first normal input area among the above-mentioned multiple normal input areas and the first normal output area among the above-mentioned multiple normal output areas corresponding to the above-mentioned first normal input area, identifying the first reconstruction critical error related to the above-mentioned first normal input area and the above-mentioned first normal output area, and the above-mentioned multiple second reconstruction errors include the above-mentioned first reconstruction critical error.

[0015] Furthermore, the above-mentioned instructions cause the above-mentioned processor to perform the following steps: splitting the above-mentioned multiple input areas and the above-mentioned multiple output areas in such a manner that the horizontal width and the vertical width of a part of the above-mentioned multiple input areas and the above-mentioned multiple output areas respectively have a first value, and splitting the above-mentioned multiple input areas and the above-mentioned multiple output areas in such a manner that at least one of the remaining horizontal widths and vertical widths has a second value smaller than the above-mentioned first value.

[0016] Furthermore, the instructions enable the processor to execute the following steps: calculating the plurality of first reconstruction errors and the threshold based on a structural similarity index metric.

[0017] Furthermore, the above-mentioned instruction causes the above-mentioned processor to execute the following steps: displaying the above-mentioned multiple first reconstruction errors associated with each of the corresponding above-mentioned multiple input areas and each of the corresponding above-mentioned multiple output areas on the display image corresponding to the above-mentioned judgment object input image, and displaying the above-mentioned abnormal area on the above-mentioned display image.

[0018] Furthermore, the instruction causes the processor to execute the following steps: identifying a third reconstruction error associated with the determination target input image and the determination target output image, and displaying the third reconstruction error on the display image.

[0019] Furthermore, the determination target input image is a captured image related to a photographic subject that repeatedly moves along a predetermined trajectory, and the abnormal area is an area where movement that deviates from a portion of the predetermined trajectory is recognized.

[0020] Effects of the Invention

[0021] The electronic device for detecting abnormalities of the present invention can obtain an output image by using a pre-learned autoencoder and segment the output image to calculate a reconstruction error, thereby improving the accuracy and reliability of the judgment of normality or non-normality using the autoencoder.

[0022] Furthermore, the electronic device for detecting anomalies of the present invention can calculate the reconstruction error by segmenting the output image without relearning the pre-learned auto-encoder, thereby improving the performance of the auto-encoder.

[0023] The specific effects of the present invention together with the above contents will be described in the following description of the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 FIG. 1 is a diagram for explaining an electronic device according to an embodiment of the present invention.

[0025] Figure 2 The flowchart is used to illustrate the operation of the processor of the electronic device according to the embodiment of the present invention.

[0026] Figure 3 and Figure 4 For illustration Figure 2 FIG. 10 is a diagram of step S200.

[0027] Figure 5 For illustration Figure 2 FIG. 5 is a diagram of step S400, step S500 and step S600.

[0028] Figure 6 For illustration Figure 2 FIG. 10 is a diagram of step S700. DETAILED DESCRIPTION

[0029] The terms or words used in this specification and the scope of protection of the invention should not be interpreted as limited to the meaning in the general or dictionary. According to the principle that the inventor can define the concept of the term or word in order to explain his invention in the best way, it should be interpreted in accordance with the meaning and concept of the technical idea of ​​the present invention. In addition, the embodiments described in this specification and the structures shown in the figures are only one embodiment of the present invention and do not represent all the technical ideas of the present invention. Therefore, when submitting this application, there may be various equivalent technical solutions that can replace them and examples that can be deformed and applied.

[0030] The terms "first", "second", "A", "B", etc. used in this specification and the scope of protection of the invention may be used to describe various structural elements, but the above-mentioned structural elements should not be limited to the above-mentioned terms. The above-mentioned terms are only used to distinguish one structural element from other structural elements. For example, without departing from the scope of protection of the present invention, the first structural element may be named as the second structural element, and similarly, the second structural element diagram may also be named as the first structural element. The term "and / or" may include a combination of multiple related recorded items or any item in multiple related recorded items.

[0031] The terms used in this specification and the scope of protection of the invention are only used to illustrate specific embodiments and do not limit the present invention. Unless otherwise defined in the context, singular expressions include plural expressions. The terms "including" or "having" in this application should be understood as not excluding in advance the existence or additional possibility of the features, numbers, steps, actions, structural elements, components or their combinations recorded in the specification.

[0032] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meanings as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0033] Terms defined in commonly used dictionaries should be interpreted as meanings consistent with the meanings of the relevant technology in the context, and should not be interpreted in an idealized or overly formalized sense unless explicitly defined in this application. In addition, the structures, processes, techniques or methods included in the various embodiments of the present invention can be shared within the scope of technical non-contradiction.

[0034] Below, we will refer to Figures 1 to 6 An electronic device for detecting anomalies (hereinafter referred to as the electronic device) according to an embodiment of the present invention will be described.

[0035] Figure 1 FIG. 1 is a diagram for explaining an electronic device according to an embodiment of the present invention.

[0036] Reference Figure 1 , the electronic device 100 of the embodiment of the present invention may include a processor 110 and a memory 120. Figure 1 In addition to the structural elements shown, the electronic device 100 may further include at least one additional structural element (eg, a communication module).

[0037] The memory 120 may store various data used by at least one structural element (e.g., the processor 110) of the electronic device 100. For example, the data may include input data or output data of software (e.g., a program) and instructions related thereto. The memory 120 may include a volatile memory or a non-volatile memory.

[0038] The memory 120 may store instructions, information or data related to the operations of the components included in the electronic device 100. For example, the memory 120 may store instructions that enable the processor 110 to perform various operations described in this specification when executed.

[0039] The processor 110 may be operatively coupled to the memory 120 to perform the overall functions of the electronic device 100. For example, the processor 110 may include more than one processor. For example, the more than one processor may include an image signal processor (ISP), an application processor (AP), or a communication processor (CP).

[0040] For example, the processor 110 can control at least one other structural element (e.g., hardware structural element or software structural element) of the electronic device 100 connected to the processor 110 by running software (e.g., program), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operation, the processor 110 can load instructions or data received from other structural elements (e.g., communication module) to the memory 120, process the instructions or data stored in the memory 120, and store the result data in the memory 120. According to one embodiment, the processor 110 may include a main processor (e.g., central processing unit or application processor) and an auxiliary processor (e.g., graphics processing unit, image signal processor, sensor hub processor or communication processor) that can operate independently or jointly. Additionally or alternatively, the auxiliary processor can be configured to use lower power than the main processor or be dedicated to a specified function. The auxiliary processor can be implemented separately from the main processor or as part of it. The program can be stored in the memory 120 as software, for example, it can include an operating system, middleware or application.

[0041] The processor 110 may communicate with the autoencoder 200 .

[0042] Will refer to Figure 2 The operation of the processor 110 will be described.

[0043] Figure 2 The flowchart is used to illustrate the operation of the processor of the electronic device according to the embodiment of the present invention.

[0044] Reference Figure 1 and Figure 2 , the processor 110 can use the pre-learned autoencoder 200 to identify normal input images and normal output images (step S100).

[0045] The processor 110 may input a normal input image to the pre-learned auto encoder 200. The normal input image may be an image that has been determined to be normal.

[0046] The autoencoder 200 may be pre-learned using at least one of a normal image or an abnormal image.

[0047] The processor 110 may receive a normal image from the outside. The processor 110 may preprocess the normal image to conform to the input format of the auto encoder 200. For example, the processor 110 may convert the normal image into a grayscale image. The processor 110 may normalize the pixel values ​​of the normal image converted into the grayscale image. For example, the processor 110 may normalize the pixel values ​​of the normal image converted into the grayscale image to a real value between 0 and 1. The processor 110 may generate a normal input image by performing additional preprocessing on the format of the normalized normal image in order to conform to the input shape of the auto encoder 200. For example, the additional preprocessing may include a process of converting the normalized normal image using a specific library.

[0048] The processor 110 may receive a normal output image from the autoencoder 200 as an output related to the normal input image.

[0049] The processor 110 may identify a threshold based on a normal input image and a normal output image (step S200). The threshold may be used to determine whether the output image of the autoencoder 200 is abnormal based on the reconstruction error. According to the calculation method of the reconstruction error, when the reconstruction error is lower than the threshold, it may be determined to be abnormal, and when the reconstruction error is higher than the threshold, it may be determined to be abnormal.

[0050] In some embodiments, the processor 110 may calculate the reconstruction error using a structural similarity index measure (SSIM). In this case, when the reconstruction error is lower than a threshold, the processor 110 may determine that it is abnormal, and when the reconstruction error exceeds the threshold, it may be determined that it is normal.

[0051] Reference Figure 3 and Figure 4 Threshold identification will be described.

[0052] Figure 3 and Figure 4 For illustration Figure 2 FIG. 10 is a diagram of step S200.

[0053] Reference Figures 1 to 4 In order to determine the threshold, the processor 110 may first convert the normal input image ( Figure 4 (a) of the input image) is divided into a plurality of normal input regions NIR (step S201). Figure 4 As shown in part (a) of , the processor 110 may divide a normal input image into a specified number of parts.

[0054] The normal input image may be divided into a plurality of normal input regions NIR. The normal input image may be divided into a specified number. Figure 4 The normal input image ( Figure 4 (a) part) is divided into 9 areas, but is not limited to this.

[0055] Some normal input areas NIR1, NIR2, NIR4, and NIR5 of the plurality of normal input areas NIR may have a first lateral width W1 and a first longitudinal width H1. For example, the first lateral width W1 and the first longitudinal width H1 may have the same value. At least one of the lateral widths and longitudinal widths of the remaining normal input areas NIR3, NIR6, NIR7, NIR8, and NIR9 of the plurality of normal input areas NIR may have a value less than the first lateral width W1 and the first longitudinal width H1. For example, the third normal input area NIR3 and the sixth normal input area NIR6 may have a second lateral width W2 and a first longitudinal width H1. The second lateral width W2 may be a value less than the first lateral width W1. For example, the seventh normal input area NIR7 and the eighth normal input area NIR8 may have a first lateral width W1 and a second longitudinal width H2. The second longitudinal width H2 may be a value less than the first longitudinal width H1. For example, the ninth normal input area NIR9 may have a second lateral width W2 and a second longitudinal width H2.

[0056] To determine the threshold, the processor 110 may convert the normal output image ( Figure 4 (b) of the embodiment of the present invention) is divided into a plurality of normal output regions NOR to correspond to a plurality of normal input regions NIR respectively (step S203). Figure 4 As shown in part (b) of FIG. 1 , the processor 110 may divide the normal output image into a specified number of parts.

[0057] The normal output image may be divided into a plurality of normal output regions NOR. The normal output image may be divided into a prescribed number. Figure 4 The normal output image ( Figure 4 The normal output image may be divided into nine regions (part (b) of the image), but is not limited thereto. The normal output image may be divided into regions corresponding to the divided regions of the normal input image (normal input regions).

[0058] Some normal output areas NOR1, NOR2, NOR4, and NOR5 of the plurality of normal output areas NOR may have a first lateral width W1 and a first longitudinal width H1. For example, the first lateral width W1 and the first longitudinal width H1 may have the same values ​​as the lateral widths and longitudinal widths of the partial normal input areas NIR1, NIR2, NIR4, and NIR5, respectively. At least one of the lateral widths and longitudinal widths of the remaining normal output areas NOR3, NOR6, NOR7, NOR8, and NOR9 of the plurality of normal output areas NOR may have a value smaller than the first lateral width W1 and the first longitudinal width H1. For example, the third normal output area NOR3 and the sixth normal output area NOR6 may have a second lateral width W2 and a first longitudinal width H1. The second lateral width W2 may be the same value as the lateral widths of the third normal input area NIR3 and the sixth normal input area NIR6. For example, the seventh normal output area NOR7 and the eighth normal output area NOR8 may have a first lateral width W1 and a second longitudinal width H2. The second longitudinal width H2 may be the same as the longitudinal widths of the seventh normal input region NIR7 and the eighth normal input region NIR8. For example, the ninth normal output region NOR9 may have a second transverse width W2 and a second longitudinal width H2.

[0059] The processor 110 may identify a plurality of second reconstruction errors associated with each of the corresponding plurality of normal input regions and each of the corresponding plurality of normal output regions (step S205). The processor 110 may identify the plurality of normal input regions by making the plurality of normal output regions correspond to each other, respectively. For example, the processor 110 may make the first normal input region NIR1 correspond to the first normal output region NOR1 and identify them as the first pairing region P1. For example, the processor 110 may make the second normal input region NIR2 correspond to the second normal output region NOR2 and identify them as the second pairing region P2. The processor 110 may identify the corresponding pairing region for each segmented normal input image and each segmented normal output image region in this manner.

[0060] The processor 110 may identify a plurality of second reconstruction errors associated with the corresponding overall normal input region and the overall normal output region by calculating the reconstruction critical error for the corresponding normal input region and the normal output region. For example, the processor 110 may identify a first reconstruction critical error associated with the first pairing region P1. For example, the processor 110 may identify a second reconstruction critical error associated with the second pairing region P2. The processor 110 may identify the reconstruction critical errors of all pairing regions in this manner. The plurality of second reconstruction errors may include the reconstruction critical errors identified for all pairing regions. For example, the plurality of second reconstruction errors may include the first reconstruction critical error and the second reconstruction critical error.

[0061] In some embodiments, the processor 110 may calculate the reconstruction critical error using a structural similarity index metric.

[0062] The processor 110 may identify a threshold value based on one of the plurality of second reconstruction errors (step S207). The processor 110 may identify the threshold value based on any one of a minimum value or a maximum value of the plurality of second reconstruction errors.

[0063] When the processor 110 calculates the reconstruction critical error using the structural similarity index metric, the processor 110 may identify the threshold based on the minimum value of the plurality of second reconstruction errors including the reconstruction critical error. For example, the processor 110 may set the value of the minimum value of the plurality of second reconstruction errors multiplied by a predetermined value as the threshold.

[0064] In some embodiments, the processor 110 may perform a first-only threshold setting process on the autoencoder 200 before receiving the decision object image. For example, the processor 110 may set the threshold based on several captured normal input images before receiving the decision object image.

[0065] When setting a threshold of reconstruction error for abnormality determination using the automatic encoder 200, the electronic device 100 of the embodiment of the present invention can segment the normal input image and the normal output image respectively, calculate the reconstruction error associated with each corresponding segmented region, and then set the threshold based on one of the multiple reconstruction errors. In this way, the electronic device 100 of the embodiment of the present invention can more accurately determine normality and abnormality.

[0066] Re-reference Figure 1 and Figure 2 , the processor 110 can use the pre-learned autoencoder 200 to identify the determination object input image and the determination object output image (step S300). After setting the threshold, the processor 110 can input the determination object input image to the pre-learned autoencoder 200, and receive the determination object output image as an output related to the determination object input image from the autoencoder 200.

[0067] The processor 110 may receive the decision object image from the outside. The processor 110 may preprocess the decision object image to conform to the input format of the auto encoder 200. For example, the processor 110 may convert the decision object image into a grayscale image. The processor 110 may normalize the pixel values ​​of the decision object image converted into the grayscale image. For example, the processor 110 may normalize the pixel values ​​of the decision object image converted into the grayscale image to a real value between 0 and 1. The processor 110 may generate the decision object input image by performing additional preprocessing on the format of the normalized decision object image in order to conform to the input shape of the auto encoder 200. For example, the additional preprocessing may include a process of converting the normalized decision object image using a specific library.

[0068] The processor 110 may divide the determination object input image into a plurality of input regions (step S400). The processor 110 may divide the determination object output image into a plurality of output regions to correspond to the plurality of input regions (step S500). The processor 110 may identify a plurality of first reconstruction errors associated with each of the corresponding plurality of input regions and each of the corresponding plurality of output regions (step S600).

[0069] Figure 5 For illustration Figure 2 FIG. 5 is a diagram of step S400, step S500 and step S600.

[0070] Reference Figure 1 , Figure 2 and Figure 5 In order to identify multiple first reconstruction errors, the processor 110 may first input the judgment object into the image ( Figure 5 (a) is divided into multiple input regions IR. Figure 5 As shown in part (a) of FIG. 1 , the processor 110 may divide the determination target input image into a predetermined number of parts.

[0071] The determination target input image may be divided into a plurality of input regions IR. The determination target input image may be divided into a predetermined number. When calculating the threshold, the determination target input image may be divided into the same number as the number of divisions of the normal input image.

[0072] When calculating the threshold, the determination target input image may be divided into regions whose horizontal width and vertical width are the same as those of each region into which the normal input image is divided.

[0073] Some input areas IR1, IR2, IR4, and IR5 of the multiple input areas IR may have a first lateral width W1 and a first longitudinal width H1. For example, the first lateral width W1 and the first longitudinal width H1 may be the same value. At least one of the lateral widths and longitudinal widths of the remaining input areas IR3, IR6, IR7, IR8, and IR9 of the multiple input areas IR may have a value smaller than the first lateral width W1 and the first longitudinal width H1. For example, the third input area IR3 and the sixth input area IR6 may have a second lateral width W2 and a first longitudinal width H1. The second lateral width W2 may be a value smaller than the first lateral width W1. For example, the seventh input area IR7 and the eighth input area IR8 may have a first lateral width W1 and a second longitudinal width H2. The second longitudinal width H2 may be a value smaller than the first longitudinal width H1. For example, the ninth input area IR9 may have a second lateral width W2 and a second longitudinal width H2.

[0074] In order to calculate the plurality of first reconstruction errors, the processor 110 may determine the output image of the object ( Figure 5 (b) is divided into a plurality of output regions OR to correspond to a plurality of input regions IR respectively. Figure 5 As shown in part (b) of FIG. 1 , the processor 110 may divide the output image into a specified number of parts.

[0075] The output image may be divided into a plurality of output areas OR. The output image may be divided into a predetermined number. When calculating the threshold, the determination target output image may be divided into the same number of divisions as the normal output image.

[0076] Partial output regions OR1, OR2, OR4, and OR5 of the plurality of output regions OR may have a first lateral width W1 and a first longitudinal width H1. For example, the first lateral width W1 and the first longitudinal width H1 may be the same value as the lateral width and longitudinal width of each partial input region IR1, IR2, IR4, and IR5. At least one of the lateral widths and longitudinal widths of the remaining output regions OR3, OR6, OR7, OR8, and OR9 of the plurality of output regions OR may have a value less than the first lateral width W1 and the first longitudinal width H1. For example, the third output region OR3 and the sixth output region OR6 may have a second lateral width W2 and a first longitudinal width H1. The second lateral width W2 may be the same value as the lateral width of the third input region IR3 and the sixth input region IR6. For example, the seventh output region OR7 and the eighth output region OR8 may have a first lateral width W1 and a second longitudinal width H2. The second longitudinal width H2 may be the same value as the longitudinal width of the seventh input region IR7 and the eighth input region IR8. For example, the ninth output region OR9 may have a second lateral width W2 and a second longitudinal width H2.

[0077] The processor 110 may identify a plurality of first reconstruction errors associated with each of the corresponding plurality of input regions and each of the corresponding plurality of output regions. The processor 110 may identify the plurality of input regions by making the plurality of output regions correspond to each other, respectively. For example, the processor 110 may correspond the first input region IR1 to the first output region OR1 and identify it as a third pairing region P3. For example, the processor 110 may correspond the second input region IR2 to the second output region OR2 and identify it as a fourth pairing region P4. The processor 110 may identify the corresponding pairing region for each segmented input image and each segmented output image region in this manner.

[0078] The processor 110 may identify a plurality of first reconstruction errors associated with the corresponding overall input region and the overall output region by calculating the reconstruction decision errors for the corresponding input region and the output region. For example, the processor 110 may identify a first reconstruction decision error associated with the third pairing region P3. For example, the processor 110 may identify a second reconstruction decision error associated with the fourth pairing region P4. The processor 110 may identify the reconstruction decision errors for all pairing regions in this manner. The plurality of first reconstruction errors may include reconstruction decision errors identified for all pairing regions. For example, the plurality of first reconstruction errors may include a first reconstruction decision error and a second reconstruction decision error.

[0079] In some embodiments, the processor 110 may calculate the reconstruction decision error using a structural similarity index metric.

[0080] Re-reference Figure 1 and Figure 2 , the processor 110 may identify an abnormal region based on each comparison result between the threshold and the plurality of first reconstruction errors (step S700). The processor 110 may identify an abnormal region in each of the corresponding plurality of input regions and each of the corresponding plurality of output regions based on each comparison result between the threshold and the plurality of first reconstruction errors.

[0081] The processor 110 may compare the threshold value and the plurality of first reconstruction errors, respectively. For example, the plurality of first reconstruction errors may include a first reconstruction decision error and a second reconstruction decision error. In this case, the processor 110 may compare the threshold value and the first reconstruction decision error, and compare the threshold value and the second reconstruction decision error.

[0082] The processor 110 may compare the reconstruction determination errors calculated for all paired regions with the threshold in this manner. The processor 110 may determine whether each paired region is normal or abnormal by comparing the reconstruction determination errors calculated for all paired regions with the threshold.

[0083] For example, when the processor 110 calculates a plurality of first reconstruction errors and a plurality of second reconstruction errors using the structural similarity index metric, the processor 110 may identify an input region and an output region having a reconstruction determination error lower than a threshold as an abnormal region. For example, when the processor 110 calculates a plurality of first reconstruction errors and a plurality of second reconstruction errors using the structural similarity index metric, the processor 110 may identify an input region and an output region having a reconstruction determination error higher than a threshold as a normal region.

[0084] Figure 6 For illustration Figure 2 FIG. 10 is a diagram of step S700.

[0085] Reference Figure 1 , Figure 2 and Figure 6 , the processor 110 may display a plurality of first reconstruction errors associated with each of the corresponding plurality of input regions and each of the corresponding plurality of output regions on a display image DI corresponding to the input image to be determined. The processor 110 may display and provide the reconstruction determination error calculated for each corresponding pairing region on the display image DI according to the pairing region. The display image DI is an image corresponding to the input image to be determined, for example, the input image to be determined, or an image preprocessed from the input image to be determined.

[0086] In the case where a plurality of first reconstruction errors are displayed on the display image DI, the processor 110 may simultaneously display whether the input region is normal or abnormal. For example, the processor 110 may display the first reconstruction judgment error associated with the first input region IR1 and the first output region OR1 on the display image DI (step 702), and simultaneously display the first input region IR1 and the first output region OR1 as normal based on the comparison result between the first reconstruction judgment error and the threshold. For example, the processor 110 may display the third reconstruction judgment error associated with the third input region IR3 and the third output region OR3 on the display image DI (step 703), and simultaneously display the third input region IR3 and the third output region OR3 as abnormal based on the comparison result between the third reconstruction judgment error and the threshold. For example, the processor 110 may display the sixth reconstruction judgment error associated with the sixth input region IR6 and the sixth output region OR6 on the display image DI (step 704), and simultaneously display the sixth input region IR6 and the sixth output region OR6 as abnormal based on the comparison result between the sixth reconstruction judgment error and the threshold.

[0087] In some embodiments, the processor 110 may identify a third reconstruction error associated with the determination object input image and the determination object output image, and display the third reconstruction error on the display image DI (step 701). That is, for comparison, the processor 110 may calculate the third reconstruction error associated with the overall determination object input image and the overall determination object output image and display it on the display image DI. According to the electronic device 100 of an embodiment of the present invention, even if the third reconstruction error is determined to be abnormal when compared with a threshold value, a part of the divided regions (e.g., the input region and the output region) of the determination object input image and the determination object output image may be determined to be normal. When calculating the reconstruction error, the electronic device 100 of an embodiment of the present invention can divide the input image and the output image of the autoencoder 200 respectively and calculate the reconstruction error associated with the divided regions, so that compared with the result of calculating the reconstruction error with the overall input image and output image as the object, it is possible to judge normal and abnormal more accurately and finely.

[0088] In some embodiments, the determination object input image may be a captured image related to a captured object moving repeatedly along a specified trajectory. Therefore, the abnormal region may be identified as moving away from a part of the trajectory in the specified trajectory. For example, Figure 6 the determination object input image may be a captured image related to a robotic arm moving to preset first to fourth coordinates. At this time, the third coordinate may be corrected from a normal coordinate to an abnormal coordinate. The processor 110 may use the captured image related to the robotic arm along the changed trajectory as the determination object input image to calculate the reconstruction determination error. As a result, the third input region IR3 and the third output region OR3 close to the third coordinate and the sixth input region IR6 and the sixth output region OR6 are determined to be abnormal. By dividing the determination object input image and the determination object output image respectively and calculating the reconstruction error of the divided regions, the electronic device 100 of an embodiment of the present invention can enable the autoencoder 200 to accurately perform local abnormality determination without re-learning the autoencoder 200.

[0089] The term "module" used in this specification may include units implemented in hardware, software, or firmware. For example, it may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrally formed component or the smallest unit or a part of the above components that execute one or more functions. For example, according to an embodiment, a module can be implemented in the form of an application-specific integrated circuit (ASIC).

[0090] Various embodiments of the present specification may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., a memory) that can be read by a device (machine) (e.g., an electronic device 100). For example, the processor 110 of the device (e.g., an electronic device 100) may call at least one of the one or more instructions stored in the storage medium and run it. This enables the device to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by an editor or code run by an interpreter. The storage medium readable by the device may be provided in the form of a non-transitory storage medium. Among them, "non-volatile" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave). The above term does not distinguish whether the data is semi-permanently stored in the storage medium or temporarily stored.

[0091] According to one embodiment, the methods of various embodiments disclosed in this specification may be included in a computer program product and provided. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or through an application store (e.g., Play Store). TM ) or directly online publishing (e.g., downloading or uploading) between two user devices (e.g., smartphones). In the case of online publishing, at least a portion of the computer program product may be temporarily stored on a device-readable storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.

[0092] According to various embodiments, each structural element (e.g., module or program) of the above-mentioned structural elements described may include a single or multiple individuals. According to various embodiments, one or more structural elements or actions in the corresponding above-mentioned structural elements may be omitted, or one or more other structural elements or actions may be added. Alternatively or additionally, multiple structural elements (e.g., modules or programs) may be integrated into one structural element. In this case, the integrated structural element may perform one or more functions of each structural element of the above-mentioned multiple structural elements in the same or similar manner as the functions performed by the corresponding structural elements in the above-mentioned multiple structural elements before the above-mentioned integration. According to various embodiments, the actions performed by modules, programs or other structural elements may be performed sequentially, in parallel, repeatedly or heuristically, or one or more actions in the above-mentioned actions may be performed in a different order, omitted, or one or more other actions may be added.

[0093] The above description is only an exemplary description of the technical idea of ​​the present embodiment. For ordinary technicians in the technical field to which the present embodiment belongs, various modifications and deformations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiment is not intended to limit the technical idea of ​​the present embodiment, but is used for illustration, and the scope of the technical idea of ​​the present embodiment is not limited to these embodiments. The protection scope of the present embodiment shall be interpreted by the attached invention claim protection scope, and all technical ideas within its equivalent range shall be interpreted as included in the invention claim protection scope of the present embodiment.

Claims

1. An electronic device for detecting anomalies, characterized in that: include: Processor; and A memory is operably connected to the processor. The memory stores instructions that enable the processor to execute the following steps when the memory is executed: Feed the normal input image into the pre-learned autoencoder, receiving a normal output image from the above autoencoder as an output associated with the above normal input image, Based on the above normal input image and the above normal output image recognition threshold, The judgment object input image is input to the above automatic encoder, receiving a decision target output image from the autoencoder as an output related to the decision target input image, The input image to be judged is divided into a plurality of input areas. The determination target output image is divided into a plurality of output regions to correspond to the plurality of input regions respectively. identifying a plurality of first reconstruction errors associated with each of the corresponding plurality of input regions and each of the corresponding plurality of output regions, Based on the respective comparison results between the threshold and the plurality of first reconstruction errors, an abnormal region in each of the corresponding plurality of input regions and each of the corresponding plurality of output regions is identified.

2. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: identifying a first input region among the plurality of input regions and a first output region among the plurality of output regions corresponding to the first input region, identifying the first reconstruction decision error included in the plurality of first reconstruction errors as a first reconstruction decision error associated with the first input region and the first output region, The threshold is compared with the first reconstruction determination error to identify whether the corresponding first input region and first output region are normal.

3. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: The above normal input image is divided into multiple normal input areas. The normal output image is divided into a plurality of normal output regions, respectively corresponding to the plurality of normal input regions. identifying a plurality of second reconstruction errors associated with each of the corresponding plurality of normal input regions and each of the corresponding plurality of normal output regions, The threshold is identified based on one of the plurality of second reconstruction errors.

4. The electronic device according to claim 3, characterized in that: The above instructions cause the above processor to execute the following steps: Calculating the plurality of first reconstruction errors, the plurality of second reconstruction errors and the threshold value based on the structural similarity index measurement, The threshold is set based on a minimum value among the plurality of second reconstruction errors.

5. The electronic device according to claim 3, characterized in that: The above instructions cause the above processor to execute the following steps: identifying a first normal input region among the plurality of normal input regions and a first normal output region among the plurality of normal output regions corresponding to the first normal input region, identifying a first reconstruction critical error associated with the first normal input region and the first normal output region, The plurality of second reconstruction errors include the first reconstruction critical error.

6. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: The plurality of input areas and a portion of the plurality of output areas are divided in such a manner that the horizontal width and the vertical width of each portion have a first value, The division is performed in such a manner that at least one of the remaining horizontal width and vertical width of the plurality of input regions and the plurality of output regions has a second value smaller than the first value.

7. The electronic device according to claim 1, characterized in that: The instructions enable the processor to execute the following steps: calculating the plurality of first reconstruction errors and the threshold based on a structural similarity index metric.

8. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: displaying the plurality of first reconstruction errors associated with each of the plurality of corresponding input regions and each of the plurality of corresponding output regions on a display image corresponding to the determination target input image, The abnormal area is displayed on the display image.

9. The electronic device according to claim 8, characterized in that: The above instructions cause the above processor to execute the following steps: identifying a third reconstruction error associated with the decision target input image and the decision target output image, The third reconstruction error is displayed on the display image.

10. The electronic device according to claim 1, characterized in that: The determination target input image is a captured image related to a photographic subject that repeatedly moves along a predetermined trajectory. The abnormal area is an area where movement deviating from a portion of the predetermined trajectory is recognized.

Images