Method and appararus of generating numerical data for deep learning using diagnosing beat of futus using artificail intelligence

Abstract

The present invention relates to a method and apparatus for image analysis using artificial intelligence. A method for analyzing images using artificial intelligence according to an embodiment of the present invention comprises, in a method for analyzing images using an artificial intelligence-based image analysis device, receiving real-time images in real time; analyzing the real-time images and all features of a first image by similarity comparison and storing first image information included in the real-time images in memory; identifying a first image clip in the first image information; extracting clip frame features from all frames of the first image clip, a previous image clip which is one image clip prior to the first image clip, and a subsequent image clip which is one image clip after the first image clip; and identifying the first image start time and the first image end time using each clip frame feature extracted from all frames of the first image clip, the previous image clip, and the subsequent image clip, along with the first image first frame feature and the first image last frame feature.

Description

Technology Field

[0001] The present invention relates to a method and apparatus for image analysis using artificial intelligence. Background Technology

[0002] Image analysis using artificial intelligence (AI) is widely utilized in various fields and relies heavily on advancements in machine learning (ML) and deep learning technologies. Conventional image analysis technology has evolved by extracting image features and performing analysis based on them.

[0003] Conventional image analysis methods primarily used fixed algorithms and rule-based pattern recognition, but the introduction of AI and deep learning has enabled more efficient and sophisticated analysis. In particular, deep learning-based technology can autonomously learn various patterns in images and recognize objects with higher accuracy in complex scenes.

[0004] However, in the past, when specific video content, such as advertisements, was inserted in the middle of a video such as a movie, it was difficult to quickly analyze information about the specific video content (e.g., name, length, start time, end time, etc.) from real-time video received from numerous channels. Prior art literature

[0005] Republic of Korea Published Patent No. 10-2023-0148667 (Published on October 25, 2023) The problem to be solved

[0006] The technical problem that the present invention aims to solve is to provide an image analysis method and apparatus using artificial intelligence used to rapidly analyze information of an image with a specific purpose in real-time video. means of solving the problem

[0007] A method for analyzing images using artificial intelligence according to an embodiment of the present invention comprises, in a method for analyzing images using an artificial intelligence-based image analysis device, receiving real-time images in real time; analyzing the real-time images and all features of a first image by similarity comparison and storing first image information included in the real-time images in memory; identifying a first image clip in the first image information; and extracting clip frame features from all frames of the first image clip, a previous image clip which is one image clip prior to the first image clip, and a subsequent image clip which is one image clip after the first image clip.

[0008] A video analysis method using artificial intelligence according to one embodiment of the present invention further includes the step of checking whether a first video verification command is received to check whether at least one of the first videos is included in the real-time video.

[0009] The step of analyzing the real-time image and the entire features of the first image by the similarity comparison and storing the first image information included in the real-time image in the memory is characterized by performing the real-time image in a sliding window manner at a predetermined specific time interval unit.

[0010] A video analysis method using artificial intelligence according to an embodiment of the present invention further includes the step of determining the start time of the first video and the end time of the first video using each clip frame feature extracted from all frames of the first video clip, the previous video clip, and the subsequent video clip, and the first frame feature and the last frame feature of the first video.

[0011] An image analysis device using artificial intelligence according to one embodiment of the present invention includes a memory for storing a real-time image, a real-time image analysis unit that receives the real-time image in real time, analyzes the real-time image and the entire features of a first image by comparing similarity, and stores first image information included in the real-time image in the memory, and a second image analysis unit that checks the first image start time and the first image end time of the first image.

[0012] The second image analysis unit is characterized by extracting clip frame features from all frames of a first image clip, a previous image clip which is one image clip prior to the first image clip, and a subsequent image clip which is one image clip after the first image clip. Effects of the invention

[0013] According to the image analysis method and apparatus using artificial intelligence according to an embodiment of the present invention, the following effects are achieved.

[0014] First, the present invention can quickly and efficiently verify a first video, such as an advertisement video, in a real-time video using artificial intelligence.

[0015] Second, the present invention does not extract frame features for every frame of every video clip, but uses the first video information of the first video, the first frame feature of the first video, and the last frame feature of the first video analyzed by the second video analysis unit to quickly, accurately, and efficiently verify the first video information, start time, end time, etc. of the first video without burdening the video analysis device.

[0016] Third, the present invention can efficiently analyze images, such as advertisements included in real-time video, by rescaling the size of the image and generating features of the same size regardless of the length of the image.

[0017] Fourth, the present invention can quickly and efficiently determine the start time and end time of the first video by using each clip frame feature extracted from all frames of the first video clip, the previous video clip which is one video clip prior to the first video clip, and the subsequent video clip which is one video clip after the first video clip, as well as the first video first frame feature and the first video last frame feature. Brief explanation of the drawing

[0018] FIG. 1 is a block diagram illustrating an image analysis device using artificial intelligence according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an image analysis method using artificial intelligence according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating a method for generating features of a first image according to an embodiment of the present invention. FIG. 4a is a diagram illustrating a method for generating all features of a first image according to an embodiment of the present invention. FIG. 4b is a diagram illustrating a method for generating all features of a first image according to another embodiment of the present invention. FIG. 5 illustrates an example in which an image analysis device using artificial intelligence according to an embodiment of the present invention detects a first image in a real-time image. Specific details for implementing the invention

[0019] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0020] Furthermore, the terms used in this specification are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms “comprises” and / or “comprising” as used in this specification do not exclude the presence or addition of other components other than those mentioned. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which the invention pertains.

[0021] Hereinafter, in order to explain the present invention more specifically, embodiments according to the present invention will be described in more detail with reference to the accompanying drawings.

[0023] FIG. 1 is a block diagram illustrating an image analysis device using artificial intelligence according to an embodiment of the present invention.

[0024] Referring to FIG. 1, an image analysis device using artificial intelligence according to one embodiment of the present invention (hereinafter referred to as the “image analysis device”) includes a memory (10), a first image analysis unit (20), a real-time image analysis unit (30), a second image analysis unit (40), and a control unit (50). These components communicate through one or more communication buses or signal lines.

[0025] Memory (10) is a part that stores various data.

[0026] The memory (10) can store real-time video, the entire first video feature, the first frame feature of the first video, the last frame feature of the first video, first video information, the first video start time, the first video end time, etc. Here, the first video information may include whether the first video exists, at least one first video clip determined to constitute the first video, the length of the first video (or the file length of the first video), the name of the first video (or the file name of the first video), location information of the first video, etc.

[0027] The memory (10) can store clip frame features extracted from all frames of video clips, including at least one video clip of the stored first video, a first video clip, a previous video clip, a video clip prior to the first video clip, and a subsequent video clip, a video clip following the first video clip.

[0028] The memory (10) can store various preset commands. For example, preset commands can store a first image verification command, a preset specific time interval unit, a preset vector size, a preset specific size, a similarity comparison method, etc.

[0029] The first image analysis unit (20) analyzes the first image to generate the first image whole features, extracts the first image first frame features and the first image last frame features, and stores the first image whole features, the first image first frame features, and the first image last frame features in memory (10).

[0030] Here, the first video is a video for a specific purpose, such as an advertisement video (or advertisement file), and can be a video for a specific purpose inserted into a live video of a movie or drama.

[0031] The first image analysis unit (20) is a part that analyzes features of the first image in advance in order to analyze the first image included in the real-time image, and can be named, for example, a pre-image analysis unit.

[0032] The first image analysis unit (20) can rescale the first image to a predetermined specific size, extract frame features from the first image for every predetermined specific number of frames, generate all frame features extracted from the first image to create the first image overall features, and extract the first image first frame feature and the first image last frame feature from the first image.

[0033] The real-time video analysis unit (30) receives real-time video in real time, obtains various information of the first video (e.g., an advertisement) included in the real-time video, and stores it in memory (10). For example, the real-time video analysis unit (30) can analyze the first video included in the real-time video using a similarity comparison method with an artificial intelligence (or artificial intelligence model) learned according to a preset algorithm.

[0034] The real-time video analysis unit (30) receives a real-time video in real time and, upon receiving a first video verification command to check whether at least one first video is included in the real-time video, analyzes the real-time video by comparing the real-time video and the entire features of the first video in a sliding window method of a pre-set specific time interval unit and stores the first video information included in the real-time video in memory (10).

[0035] The second video analysis unit (40) is a part that checks the first video start time and the first video end time of the first video.

[0036] The second video analysis unit (40) checks the first video clip in the first video information and checks the first video start time and the first video end time using each clip frame feature extracted from all frames of the first video clip, the previous video clip which is one video clip before the first video clip, and the subsequent video clip which is one video clip after the first video clip, as well as the first video first frame feature and the first video last frame feature.

[0037] The control unit (50) is a part that controls the image analysis device using artificial intelligence according to the present invention. In addition, the control unit (50) is a part that controls each component of the image analysis device using artificial intelligence.

[0038] The control unit (50) can control the uploading of the first image (or first image file) to the first image analysis unit (20).

[0039] The control unit (50) can control various commands according to user commands.

[0040] The display unit (60) provides an interface for visual output between the user and the image analysis device.

[0041] The display unit (60) can display various data including real-time video, first video, first video information, etc.

[0042] The user can input a command signal to the image analysis device to change pre-set commands, etc., through an input unit (not shown).

[0043] However, in some embodiments such as a touchscreen, the display part (60) and the input part may be integrated or separated.

[0045] FIG. 2 is a block diagram illustrating an image analysis method using artificial intelligence according to an embodiment of the present invention.

[0046] FIG. 5 illustrates an example in which an image analysis device using artificial intelligence according to an embodiment of the present invention detects a first image in a real-time image.

[0047] Referring to FIGS. 2 and FIGS. 5, an image analysis method using artificial intelligence according to an embodiment of the present invention will be described.

[0048] The image analysis device analyzes the first image to generate the first image whole features, extracts the first frame features of the first image and the last frame features of the first image, and stores the first image whole features, the first frame features of the first image, and the first frame features of the first image in memory (S201).

[0049] Here, the first video is a video for a specific purpose, such as an advertisement video (or advertisement file), and can be a video for a specific purpose inserted into a live video of a movie or drama.

[0050] Hereinafter, a method for generating features of the first image will be explained in detail with reference to FIG. 3.

[0051] The video analysis device receives real-time video in real time (S202). Additionally, the video analysis device stores the real-time video received in real time in memory. Here, the real-time video is a real-time video received through numerous channels, and various types of video, such as broadcast streams and movie streams, are possible.

[0052] The image analysis device checks whether it has received a first image verification command to check whether at least one first image is included in the real-time image (S203).

[0053] When the image analysis device receives a first image verification command to check whether at least one first image is included in the real-time image, it analyzes the real-time image and the first image by comparing the entire features of the real-time image and the first image and stores the first image information included in the real-time image in memory (S204).

[0054] The image analysis device can perform analysis of real-time images by comparing the similarity between the real-time image and all features of the first image using a sliding window method in units of specific time intervals set in advance.

[0055] Here, the entire feature of the first image refers to the entire feature of the first image stored in memory with a preset vector size. At this time, the preset vector size can be various embodiments such as 2048 vector values, 1028 vector values, etc.

[0056] In addition, the pre-set specific time interval unit is a time that the user can set, for example, between 1 second and 60 seconds.

[0057] For example, if a specific time interval unit is set to 15 seconds, the video analysis device can compare similarity of 15-second video clips in a sliding window manner in 5-second intervals.

[0058] Additionally, the first image information may include the existence of the first image, at least one first image clip determined to constitute the first image, the length of the first image (or file length of the first image), the name of the first image (or file name of the first image), location information of the first image, etc.

[0059] If the image analysis device does not receive a first image verification command to check whether at least one first image is included in the real-time image, it performs step S203.

[0060] In the present invention, similarity comparison can be performed using the following mathematical formula 1.

[0061]

[0062] Similarity, or cosine similarity, is a method for measuring how similar two vectors are in a multidimensional space.

[0063] The cosine similarity between two vectors A and B can be defined as the cosine value of the angle θ between these vectors.

[0064] Here, AB is the inner product of vectors A and B, and ∥A∥ and ∥B∥ are the magnitudes (or lengths) of vectors A and B, respectively.

[0065] The similarity between vectors measured using the cosine value of the angle between two vectors in an inner product space means that if the cosine similarity is 1, the two vectors point in the same direction and are completely identical; if the cosine similarity is 0, the two vectors are orthogonal to each other, independent, or have no similarity; and if the cosine similarity is -1, the two vectors point in completely opposite directions.

[0066] In addition to cosine similarity, the present invention allows for the application of similarity comparison methods of various embodiments.

[0067] The image analysis device identifies the first image clip in the first image information (S205). Here, the first image clip may consist of at least one clip.

[0068] The image analysis device extracts clip frame features from all frames of the first image clip, the previous image clip which is one image clip prior to the first image clip, and the subsequent image clip which is one image clip after the first image clip (S206).

[0069] The video analysis device determines the start time of the first video and the end time of the first video using each clip frame feature extracted from all frames of the first video clip, the previous video clip which is one video clip before the first video clip, and the subsequent video clip which is one video clip after the first video clip, as well as the first frame feature of the first video and the last frame feature of the first video (S207).

[0070] For example, the video analysis device can determine the start time and end time of the first video by comparing the similarity between each clip frame feature extracted from all frames of the first video clip, the previous video clip which is one video clip prior to the first video clip, and the subsequent video clip which is one video clip after the first video clip, and the first frame first frame feature of the first video.

[0071] The video analysis device can extract clip frame features from all frames of video clips including at least one video clip of a first video stored in memory in a specific time unit (e.g., 15 seconds), a previous video clip which is one video clip before the first video clip, and a subsequent video clip which is one video clip after the first video clip, and compare the extracted clip frame features with the first video first frame feature and the first video last frame feature to determine the accurate start time and end time of the first video.

[0072] For example, if there are three first video clips, the video analysis device can extract clip frame features from a total of five video clips, including one previous video clip and one subsequent video clip, and compare the first frame feature of the first video and the last frame feature of the first video to determine the accurate start time of the first video and the end time of the first video.

[0073] By doing this, the present invention can quickly and accurately check the first image information, the first image start time, the first image end time, etc. of the first image without burdening the image analysis device by using the first image information, the first frame feature, and the last frame feature of the first image analyzed by the second image analysis unit (40), without extracting frame features for every frame of every image clip.

[0074] Referring to Fig. 5, an example is shown of detecting an advertisement video, which is a first video, in a real-time broadcast stream, which is a real-time video.

[0075] For example, the video analysis device can quickly and accurately identify and provide information to the user regarding seven advertisements in a real-time broadcast stream, such as an SK Telecom advertisement for 30 seconds between 2:00 and 2:30, an Albamon advertisement for 15 seconds between 4:00 and 4:15, a Keunmam Halmae advertisement for 120 seconds between 6:00 and 8:00, a Tbroad advertisement for 30 seconds between 10:00 and 10:30, a Paldo Wang Ttukkeong advertisement for 15 seconds between 12:00 and 12:15, a Uya advertisement for 30 seconds between 14:00 and 14:30, and an Indian advertisement for 30 seconds between 16:00 and 16:30.

[0077] FIG. 3 is a block diagram illustrating a method for generating features of a first image according to an embodiment of the present invention.

[0078] FIG. 4a is a drawing for explaining a method for generating all features of a first image according to one embodiment of the present invention, and FIG. 4b is a drawing for explaining a method for generating all features of a first image according to another embodiment of the present invention.

[0079] Referring to FIGS. 3 to 4b, a method for generating features of a first image according to an embodiment of the present invention will be described.

[0080] The image analysis device rescales the first image to a preset specific size (S301). Here, the preset specific size can be set by the user, for example, the preset specific size can be 640*360. This is so that the present invention reduces the size of the image for fast real-time processing of the first image.

[0081] The image analysis device extracts frame features from the first image for a predetermined number of frames (S302). At this time, the size of the frame features can be a predetermined number of vector values ​​regardless of the length of the first image. For example, the size of the frame features can be 2048 vector values ​​regardless of the length of the first image.

[0082] For example, the image analysis device can extract features for one frame every six frames of the first image.

[0083] The image analysis device generates the total features of the first image by summing all frame features extracted from the first image (S303). At this time, the image analysis device generates the total features of the first image by summing the vector values ​​of each frame feature.

[0084] Referring to Fig. 4a, Fig. 4a is an example where the first video is a 10-second advertisement video. In this case, each feature consists of 2028 vector values. For example, if the first feature (vector value) of the first frame from which the first feature was extracted is 3, 1, 7, …, 2, 6, 4, and the sixth feature (vector value) of the sixth frame from which the second feature was extracted is 3, 2, 6, …, 1, 5, 3, then the total features (vector value) become 6, 3, 13, …, 3, 11, 7.

[0085] Referring to FIG. 4b, FIG. 4b is an example where the first video is a 20-second advertisement video. In this case, each feature consists of 2028 vector values, identical to the 10-second advertisement video in FIG. 4a. For example, the first feature (vector value) of the first frame from which the first feature was extracted is 1, 3, 7, …, 4, 2, 6; the sixth feature (vector value) of the sixth frame from which the second feature was extracted is 2, 4, 6, …, 3, 1, 5; the eleventh feature (vector value) of the eleventh frame from which the third feature was extracted is 2, 3, 5, …, 3, 1, 5; and the sixteenth feature (vector value) of the sixteenth frame from which the fourth feature was extracted is 1, 2, 3, … If , 2, 3, 1, the total features (vector values) become 6, 12, 21, … , 12, 7, 17.

[0086] The image analysis device extracts the first frame feature of the first image and the last frame feature of the first image from the first image (S304). By using the respective clip frame features extracted from all frames of the first image clip, the previous image clip which is one image clip prior to the first image clip, and the subsequent image clip which is one image clip after the first image clip, as well as the first frame feature of the first image and the last frame feature of the first image, the start time of the first image and the end time of the first image can be quickly and efficiently determined.

[0087] In this way, the present invention can quickly and efficiently verify a first video, such as an advertisement video, in a real-time video using artificial intelligence.

[0088] A method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable recording medium. A computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination. The programs (program instructions) recorded on the recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROMs; RAMs; flash memory; and hardware devices specifically configured to store and execute program instructions. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above-mentioned hardware devices may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0089] Although the present invention has been described above with reference to the embodiments illustrated in the drawings, this is merely for the purpose of explaining the invention, and those skilled in the art will understand that various modifications or equivalent embodiments are possible from the detailed description of the invention. Accordingly, the true scope of the rights of the present invention should be determined by the technical concept of the claims. Explanation of the symbols

[0090] 10: Memory 20: 1st Video Analysis Department 30: Real-time Video Analysis Department 40: 2nd Video Analysis Department 50: Control unit 60: Display unit

Claims

Claim 1 A method for an image analysis device to analyze an image comprises: receiving a real-time image in real time; checking whether a first image verification command to verify whether at least one first image is included in the real-time image has been received; analyzing the real-time image and the entire features of the first image by similarity comparison using artificial intelligence learned according to a preset algorithm to store the first image information included in the real-time image in memory; verifying a first image clip in the first image information; extracting clip frame features from all frames of the first image clip, a previous image clip which is one image clip prior to the first image clip, and a subsequent image clip which is one image clip after the first image clip; and verifying the first image start time and the first image end time using each of the clip frame features extracted from all frames of the first image clip, the previous image clip, and the subsequent image clip, along with the first image first frame feature and the first image last frame feature. A method for generating features of the first image comprises: rescaling the first image to a preset specific size; and the first image to a preset specific A method for analyzing images using artificial intelligence, comprising the steps of: extracting frame features for each number of frames; summing all frame features extracted from the first image to generate all features of the first image; and extracting the first frame feature of the first image and the last frame feature of the first image from the first image. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete

Images