Image search device, image search program, and recording medium

Abstract

To provide an image retrieval device capable of detecting that a face image of a user is leaked on the Internet with a simple configuration surely and quickly in an initial stage, an image retrieval method, an image retrieval program, and recording media.SOLUTION: An image retrieval device includes: an input / output and image comparison unit 13 for storing user's own face image data registered by a user and a keyword registered as a cause of leakage of a face image; and an image extraction unit 17b for extracting a plurality of still image data from moving image data to acquire image data including face information from the extracted still images data. The input / output and image comparison unit compares the image data including the face information acquired by the image extraction unit with the face image data stored in the input / output and image comparison unit to extract a URL of the face image data having high similarity. The image extraction unit extracts the still image data from the moving image data using a plurality of processing procedures, and integrates the still image data extracted by the plurality of processing procedures.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image search device for searching for user images that have been leaked onto the Internet. , painting The present invention relates to an image retrieval program and a recording medium having an image retrieval program recorded thereon. [Background technology]

[0002] In recent years, the number of cases of voyeurism and revenge pornography has been increasing due to the increasing performance of cameras and the widespread use of smartphones. In addition, since it is easy to post photos and videos through adult sites and SNS (social networking services), the posted photos and videos of the users themselves may be uploaded without the users' permission and leaked to sexually explicit sites on the Internet.

[0003] Patent Document 1 discloses a technique for searching for a personal image of a client that has been published on the Internet against the client's will. This image search technique extracts feature amount data of human parts from all published images on the Internet and also extracts feature amount data of the client's personal image, and searches for the client's personal image by collating these extracted feature amount data with each other to determine whether the client's personal image and multiple published images are the same image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5150572 Summary of the Invention [Problem to be solved by the invention]

[0005] The image search technology described in Patent Document 1 searches the entire Internet to check whether an image of the requester exists. However, the amount of images posted on the Internet is enormous, and extracting parts of people from all the public images and extracting and comparing the features from the extracted parts of people not only requires a significant amount of computing resources but also a significant amount of time, making it extremely difficult to realize an image search with a simple configuration.

[0006] Furthermore, since the image search technology described in Patent Document 1 is a search technology intended to prevent the violation of the client's privacy, it was not possible to quickly detect the leakage of the client's facial image, especially in the early stages, in order to prevent the client from becoming a victim of voyeurism or revenge pornography.

[0007] Therefore, an object of the present invention is to provide an image search device that can reliably detect, with a simple configuration, that a user's face image has been leaked onto the Internet. , painting The present invention provides an image retrieval program and a recording medium.

[0008] Another object of the present invention is to provide an image search device capable of quickly detecting, at an early stage, that a user's face image has been leaked onto the Internet. , painting The present invention provides an image retrieval program and a recording medium. [Means for solving the problem]

[0009] According to the present invention, there is provided an image search device including an input / output and image comparison unit that stores face image data representing a user's own face image registered by the user and a keyword registered by the user as a cause of the face image leak, and an image extraction unit that extracts a plurality of still image data from video data acquired from a WEB site that is a sexual site on the Internet and includes the keyword registered by the user, and acquires image data including face information from the extracted still image data. The input / output and image comparison unit is configured to compare the image data including face information acquired by the image extraction unit with the user's registered face image data stored in the input / output and image comparison unit, and extract a URL (web page address) of face image data having a high similarity, and the image extraction unit is configured to extract still image data from the video data using a plurality of mutually different processing procedures, and to integrate the still image data extracted by the plurality of mutually different processing procedures.

[0010] In the present invention, video data is acquired from a website that includes a keyword registered as a cause of face image leakage. Only websites for preventing voyeurism and revenge pornography are searched for without searching the entire Internet. This allows the user's face image leaked onto the Internet to be detected quickly at an early stage and simplifies the configuration of the image search device. In addition, in the present invention, when extracting a plurality of still image data from the acquired video data and acquiring image data including face information from the extracted still image data, the still image data is extracted from the video data using a plurality of different processing procedures, and the still image data extracted by the plurality of different processing procedures is integrated. In this way, the final image obtained differs depending on the processing procedure, so that the reliability of face image detection is significantly higher than when face images are extracted using only one procedure. This allows reliable detection of a user's face image leaked onto the Internet with a simple configuration.

[0011] It is preferable that the multiple different processing procedures performed by the image extraction unit include a process of playing back video data at high speed and extracting still image data that is determined by a machine learning model to contain a face, and a process of playing back video data at high speed and extracting still image data at any fixed frame interval.

[0012] It is also preferable that the multiple different processing procedures by the image extraction unit include a process of playing back the video data at high speed and extracting a still image when the machine learning model determines that a face is captured; a process of playing back the video data at high speed and extracting a still image of the face at the timing when the machine learning model determines that the face is most captured; a process of playing back the video data at high speed and extracting facial features at the timing when the machine learning model determines that the face is most captured, thereby extracting a still image of the face; and a process of playing back the video data at high speed, extracting still images of the entire video at regular frame intervals, and extracting a still image that is most likely to capture a face using the machine learning model for each extracted still image.

[0013] It is also preferable that the image extraction unit is configured to compare a confidence value of image data including face information with a threshold value and extract image data which contains the most facial images.

[0014] In this case, it is more preferable that the image extraction unit is configured to divide the image data containing facial information into a predetermined number of pieces from the beginning, select the best image data from the predetermined number of divided image data, and compare the confidence value of the selected image data with a threshold value.

[0015] It is also preferable that the device further includes a URL crawler unit that collects URLs of websites that are sexually explicit sites on the Internet and that contain keywords registered by the user, and the image extraction unit is configured to obtain video data from websites that correspond to the URLs collected by the URL crawler unit.

[0016] It is also preferable that the input / output and image comparison unit is configured to store the facial image data and the keyword transmitted from the user terminal, and configured to transmit the extracted URL to the user terminal.

[0017] According to the present invention, there is further provided an image search method for storing face image data representing a user's own face image registered by a user and a keyword registered by the user as a cause of the face image leak, extracting a plurality of still image data from video data acquired from a WEB site on the Internet that is a sexual site and includes the keyword registered by the user, and acquiring image data including face information from the extracted still image data. The acquired image data including the face information is compared with the face image data registered by the user, a URL of face image data having a high similarity is extracted, still image data is extracted from the video data using a plurality of different processing procedures, and the still image data extracted by the plurality of different processing procedures is integrated.

[0018] It is preferable that the multiple mutually different processing procedures include a process of playing back video data at high speed and extracting still image data determined by a machine learning model to contain a face, and a process of playing back video data at high speed and extracting still image data at any fixed frame interval.

[0019] It is also preferable that the multiple different processing procedures include a process of playing back the video data at high speed and extracting a still image when a machine learning model determines that a face is shown; a process of playing back the video data at high speed and extracting a still image of the face at the timing when the machine learning model determines that the face is most shown; a process of playing back the video data at high speed and extracting facial features at the timing when the machine learning model determines that the face is most shown, thereby extracting a still image of the face; and a process of playing back the video data at high speed, extracting still images of the entire video at regular frame intervals, and extracting a still image that is most likely to show a face using the machine learning model for each extracted still image.

[0020] It is also preferable to compare the confidence value of image data containing face information with a threshold value to extract image data that contains the most facial images.

[0021] In this case, it is more preferable to divide the image data containing facial information into a predetermined number of parts from the beginning, select the image data with the highest accuracy among the predetermined number of divided image data, and compare the confidence value of the selected image data with a threshold value.

[0022] It is also preferable to collect URLs of websites on the Internet that are sexually explicit and contain keywords registered by the user, and to obtain video data from the sites that correspond to the collected URLs.

[0023] It is also preferable to store the facial image data and keywords transmitted from the user terminal and transmit the extracted URL to the user terminal.

[0024] According to the present invention, there is further provided a program for causing a computer to function as an input / output and image comparison means for storing face image data representing the user's own face image registered by the user and a keyword registered by the user as the cause of the face image leak, and an image extraction means for extracting a plurality of still image data from video data acquired from a WEB site that is a sexual site on the Internet and includes a keyword registered by the user, and acquiring image data including face information from the extracted still image data. The input / output and image comparison means is a means for comparing the acquired image data including the face information with the face image data registered by the user, and extracting a URL of face image data having a high similarity, and the image extraction means is a means for extracting still image data from video data using a plurality of mutually different processing procedures, and integrating the still image data extracted by the plurality of mutually different processing procedures.

[0025] According to the present invention, there is further provided a computer-readable recording medium having recorded thereon a program for causing a computer to function as an input / output and image comparison means for storing face image data representing the user's own face image registered by the user and a keyword registered by the user as the cause of the face image leak, and an image extraction means for extracting a plurality of still image data from video data acquired from a WEB site that is a sexual site on the Internet and includes a keyword registered by the user, and acquiring image data including face information from the extracted still image data. The input / output and image comparison means is a means for comparing image data including the acquired face information with the stored face image data registered by the user, and extracting a URL of face image data having a high similarity, and the image extraction means is a means for extracting still image data from video data using a plurality of mutually different processing procedures, and integrating the still image data extracted by the plurality of mutually different processing procedures. Effect of the Invention

[0026] In the present invention, since only websites for preventing the damage caused by voyeurism and revenge pornography are searched, it is possible to quickly detect a user's face image leaked onto the Internet at an early stage and the configuration of the image search device is simplified. In addition, since the final image obtained differs depending on the processing procedure, the reliability of face image detection is significantly increased, so that it is possible to reliably detect that a user's face image has been leaked onto the Internet with a simple configuration. [Brief description of the drawings]

[0027] [Figure 1] 1 is a block diagram illustrating an overall configuration of an image search device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram illustrating the function of the image search device in FIG. [Diagram 3] 4 is a flowchart showing the processing operation of an input / output and image comparison unit in the image search device of FIG. [Figure 4] 4 is a flowchart showing the processing operation of an input / output and image comparison unit in the image search device of FIG. [Diagram 5] 4 is a flowchart showing the processing operation of a URL crawler unit in the image search device of FIG. [Figure 6] 4 is a flowchart showing a processing operation of a still image crawler unit in the image search device of FIG. [Figure 7] 4 is a flowchart showing the processing operation of a video crawler unit in the image search device of FIG. [Figure 8] 13 is a flowchart showing a partial modification of the processing operation of the video crawler unit in the image search device of FIG. [Figure 9] FIG. 13 is a diagram illustrating a registration screen for a user's own face image on a user terminal. [Figure 10] FIG. 13 is a diagram illustrating a keyword registration screen on a user terminal. [Figure 11] FIG. 13 is a diagram illustrating a search result display screen on a user terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] FIG. 1 shows an outline of the overall configuration of an image search device according to one embodiment of the present invention, and FIG. 2 explains the functions of the image search device of this embodiment.

[0029] In Figure 1, 10 denotes a user terminal, such as a smartphone, tablet terminal, or computer terminal operated by a user, on which the image search app of this embodiment is installed, 11 denotes an image search server on the cloud that can communicate with the user terminal 10, and 12 denotes the Internet that can be accessed by the image search server 11.

[0030] As shown in FIG. 1, the image search server 11 includes at least an input / output and image comparison unit 13, a cloud server 14, a URL crawler unit 15, a still image crawler unit 16, and a video crawler unit 17.

[0031] 2, the input / output and image comparison unit 13 includes a database (AWS / RDS) 13a and a storage (AWS / S3) 13b provided by a cloud computing service (AWS). The input / output and image comparison unit 13 is communicatively connectable to the user terminal 10, and is configured to store the user's facial image data transmitted from the user terminal 10 and the keywords registered by the user in association with the facial image data. The input / output and image comparison unit 13 is further configured to store the URL of a video containing a facial image having a high similarity to the user's facial image, extracted by a cloud server (Azure) 14 provided by the cloud computing service, in association with the user.

[0032] As described above, the user's face image data transmitted from the user terminal 10 is stored in the storage (AWS / S3) 13b, and the keywords registered by the user are stored in the database (AWS / RDS) 13a. The cloud server (Azure) 14 is configured to learn the images collected by the still image crawler unit 16 or the video crawler unit 17, assign a Face ID to each image, and compare the collected images with images registered by the user to determine the degree of similarity between the images.

[0033] The URL crawler unit 15 uses the library Selenium, which is an automatic crawling program that utilizes one of the Web crawlers present on the AWS cloud server (AWS / EC2) 15a. This URL crawler unit 15 is configured to collect URLs of videos containing registered keywords from a number of predetermined websites on the Internet 12 that handle sexually-charged videos and that may contain face images. Examples of the predetermined websites include Pornhub, FC2, Tokyo Motion, Twitter Video Tools, XVIDEOS, and Twitter. A list of the collected URLs is stored in a URL list unit 15b in the URL crawler unit 15.

[0034] The URL list section 15b is composed of a database (AWS / RDS) by AWS and a storage section of a local file (a configuration file in a local program file), and the collected URLs are stored in this RDS and the local file storage section. The reason for writing to the RDS is that there is a request from various systems such as a video crawler and a face similarer to access information such as URLs and video image data linked to them.

[0035] In this embodiment, the still image crawler unit 16 is mainly composed of an AWS storage (AWS / S3) 16a, an image search unit 16b, and a machine learning unit 16c.

[0036] The image search unit 16b uses a search engine described in a configuration file in a local program file to perform an image search on a website using a specified keyword stored in the input / output storage unit, and stores the displayed images in the storage (AWS / S3) 16a. Google, Yandex, Yahoo, Baidu, etc. are mainly used as search engines.

[0037] The machine learning unit 16c is constructed using a library, the FaceBoxes model, that utilizes a deep learning algorithm and is specialized for human face detection. This machine learning model learns about human facial features from a huge number of face images, and enables the detection of facial information contained in input unknown image data. The detected face information is accompanied by a numerical value called confidence, which indicates the certainty that a face is contained in the detected area. In other words, the higher this numerical value, the higher the possibility that the detected area is a face. Note that the FaceBoxes model is a publicly available machine learning model library that is constructed by learning Western face images, and therefore, as described later, a model that has been fine-tuned with Asian face images is used to improve the detection accuracy of face images compared to the original model.

[0038] The machine learning unit 16c is further configured to store all image data whose confidence values ​​exceed a predetermined threshold in the storage (AWS / S3) 16a.

[0039] In this embodiment, the video crawler unit 17 is mainly composed of a video crawler present on the cloud server (AWS / EC2) 15a, storage by AWS (AWS / S3) 17a, an image extraction unit 17b, and the aforementioned machine learning unit 16c.

[0040] The video crawler uses a search engine (Google Chrome) described in a configuration file in a local program file to play many videos at URLs stored in the URL list section 15b, and acquires a huge number of still image data for each video data. Then, from the acquired image group, it acquires only images containing face information using a FaceBoxes model library specialized for human face detection, and is configured to select one image with the highest confidence value (one image in which the face is most clearly visible).

[0041] As a method for acquiring still images from a large number of moving images at URLs stored in the URL list unit 15b, there are a method for downloading and playing the moving images, and a method for crawling and playing the web site, and either method may be used. The image extraction unit 17b plays the acquired large number of moving images and acquires a large number of still images.

[0042] The machine learning unit 16c is constructed using a library FaceBoxes model specialized for human face detection using a deep learning algorithm. This machine learning model learns about human facial features from a huge number of face images, and enables detection of facial information contained in input unknown image data. The detected facial information at this time is accompanied by a numerical value called confidence, which indicates the certainty that a face is included in the detected area. In other words, the higher this numerical value, the more likely the detected area is to be a face. Note that the FaceBoxes model is a library of machine learning models that is publicly available, and is constructed by learning Western face images, so that the face detection accuracy for Asians is not very high. Therefore, in this embodiment, fine tuning is performed by inputting independently collected Asian face images as learning data into this FaceBoxes model again to adjust the weight of the model. This improves the detection accuracy of face images compared to the original model.

[0043] The machine learning unit 16c is further configured to store all image data whose confidence values ​​exceed a predetermined threshold in the storage (AWS / S3) 17a.

[0044] 3 and 4 show the processing operation of the input / output and image comparison unit 13, and the processing operation of this input / output and image comparison unit 13 will be explained below with reference to these figures.

[0045] First, a user registers his / her own face image via the user terminal 10. That is, as shown in FIG. 9, a plurality of face image data including not only a straight face but also a smiling face and a profile is transmitted to the image search server 11. Furthermore, as shown in FIG. 10, keywords indicating "who", "when", "where", "what was done", etc. are transmitted to the image search server 11 via the user terminal 10. For example, keywords such as "ex-boyfriend", "classmate", "listener", "private photography" etc. as the photographer (if known), keywords such as "JD", "female college student", "office worker", "delivery health", "sugar daddy" etc. as the status at the time of photography (if known), keywords such as "hotel", "Shibuya", "Shinjuku", "store name", "live chat", "amateur photography", "voyeurism" etc. as the photography location (if known), keywords such as "matching app", "SNS", "Twitter", "cosplay", keywords such as the user's stage name and nickname are transmitted to the image search server 11.

[0046] As shown in FIG. 3, the input / output and image comparison unit 13 receives a plurality of pieces of face image data transmitted from the user terminal 10 and a keyword transmitted together with the face image data (step S1).

[0047] Next, the input / output and image comparison unit 13 stores the received user's facial image data in storage (AWS / S3) 13b, and stores the keywords registered by the user in association with this in database (AWS / RDS) 13a (step S2).

[0048] Furthermore, keywords that are similar to the keywords registered by the user and that are determined automatically or manually to be appropriate by crawl processing are stored in the database (AWS / RDS) 13a (step S3).

[0049] 4, the input / output and image comparison unit 13 outputs the ID of a face image with high similarity by inputting the face image of the user stored in the storage (AWS / S3) 13b to Azure Similar of the cloud server 14. Based on this ID, the database (AWS / RDS) 13a is referenced, and the URL of the video in which the face image exists and the probability that it is the person in question are extracted (step S11). That is, the URL and the probability are output from Azure, and the posting date and time are read and output from the database (AWS / RDS).

[0050] Next, the input / output and image comparison unit 13 transmits the thus obtained URL of the video containing the highly similar face image, the probability that the person is the same person, the posted date and time, and the discovered date and time to the user terminal 10 (step S12). The user terminal 10 thereby displays the search result (discovery date, whether or not this information was checked, posted date, URL, and identity probability) as shown in FIG.

[0051] FIG. 5 shows the processing operation of the URL crawler unit 15, and the processing operation of this URL crawler unit 15 will be explained below with reference to this figure.

[0052] First, the URL crawler unit 15 determines in advance a number of websites to be accessed (step S21).

[0053] Next, for the determined multiple websites, URL crawling is performed based on the keywords registered by the user and similar keywords stored in the database (AWS / RDS) 13a, and URLs of videos containing these keywords are obtained (step S22).

[0054] Next, the acquired URLs are stored as a URL list in the URL list section 15b (step S23).

[0055] FIG. 6 shows the processing operation of the still image crawler unit 16, and the processing operation of this still image crawler unit 16 will be explained below with reference to this figure.

[0056] The still image crawler unit 16 uses a predetermined search engine in the image search unit 16b to perform an image search for still images using a specified keyword stored in the database (AWS / RDS) 13a (step S31). Since there are a wide variety of websites from which images are collected, the specified keyword, for example, "secretly photographed" is used for the search in the case of secretly photographed images.

[0057] Next, the retrieved still images are temporarily stored in a local cache memory (step S32).

[0058] Next, a deep learning machine learning algorithm for face detection (e.g., the FaceBoxes model) is applied to the numerous still images stored in the cache memory to extract still images that contain faces and confidence values ​​that represent the probability that a face is included (step S33).

[0059] Next, still images (image data) whose extracted confidence values ​​exceed a threshold value are stored in the storage (AWS / S3) 16a, and a list registration process is performed (step S34).

[0060] Meanwhile, asynchronously with the processing of steps S31 to S34, when a user registers his / her own facial image data and keywords, Azure's Find Similar is called, and the image is trained in this Azure Find Similar model, while the URL of the image source, the posting date and time, keywords, the ID of the image trained in Azure, etc. are recorded in database (AWS / RDS) 13a, and the image data is uploaded to storage (AWS / S3) 13b for storage (step S35).

[0061] FIG. 7 shows the processing operation of video crawler unit 17, and the processing operation of video crawler unit 17 will be explained below with reference to this figure.

[0062] First, the video crawler unit 17 uses a predetermined image search engine (Google Chrome) to acquire a large number of videos using a specified keyword from the URLs stored in the URL list unit 15b (step S41).

[0063] Next, the acquired large number of moving images are played back at high speed to acquire still images that show the face determined by the machine learning model (step S42).

[0064] Meanwhile, the acquired large number of moving images are played back at high speed, and still images are acquired at any fixed frame interval (step S43).

[0065] Next, the many still images extracted in the processing procedure of step S42 (a processing procedure different from step S43) and the many still images extracted in the processing procedure of step S43 (a processing procedure different from step S42) are integrated and temporarily stored in a local cache memory (step S44). In this manner, in this embodiment, still images are extracted from a moving image using two mutually different processing procedures, and the still images extracted by these two processing procedures are integrated to extract still images. Since the final image obtained differs depending on the processing procedures, the reliability of face image detection is significantly increased compared to when face images are extracted using only one procedure.

[0066] Since the amount of image data is too large to record directly in the storage, in steps S42 and S43, only images that include faces are selected from a group of images that do not necessarily include faces, and the image with the highest confidence value is ultimately retained. In step S44, similarly, the image with the best image of the face (highest confidence value) is ultimately retained from the group of images that include faces. This is ultimately uploaded to the storage, but more precisely, after the image with the highest confidence value for each video listed in the URL crawler has been acquired, it is trained in Azure through a list registration process and stored in storage (AWS / S3) 17a.

[0067] When acquiring the image with the highest confidence value, it is preferable to divide a group of images obtained from a video into a predetermined number of groups (for example, every 100 images) from the beginning and select the image with the highest confidence value from among them, instead of evaluating all images. Then, for the face images obtained from each group of images, the one with the highest confidence value is finally selected. The reason for dividing into image groups and evaluating each division unit instead of selecting the image with the highest confidence value from all images is that the processing can be stopped midway if a threshold is exceeded, so that the processing can be performed efficiently overall. For example, when selecting the image with the highest confidence value from 1000 images, it is necessary to evaluate the image data of 1000 images simply, but if the image is divided into 10 groups of 100 images, if the threshold is exceeded at the 10th image, for example, it will be sufficient to evaluate 10 x 10 = 100 images.

[0068] After processing in step S44, a deep learning machine learning algorithm for face detection (e.g., the FaceBoxes model in the library) is applied to the numerous still images stored in the cache memory to extract still images containing faces and confidence values ​​representing the probability that a face is included (step S45).

[0069] Next, still images (image data) whose confidence values ​​exceed the threshold value are stored in the storage (AWS / S3) 17a (step S46).

[0070] Meanwhile, asynchronously with the processing of steps S41 to S46, when a user registers his / her own facial image data and keywords, Azure's Find Similar is called, and the image is trained in this Azure Find Similar model, while the URL of the image source, the posting date and time, keywords, the ID of the image trained in Azure, etc. are recorded in database (AWS / RDS) 13a, and the image data is uploaded to storage (AWS / S3) 13b for storage (step S47).

[0071] FIG. 8 shows a modified processing operation in which part of the processing operation of video crawler section 17 is modified.

[0072] In this modified embodiment, instead of the processing operation in step S42 of FIG. 7, the following processes are performed in parallel: as shown in step S42a of FIG. 8, a process of playing back the multiple acquired videos at high speed and using a machine learning model to extract a still image when it is determined that a face is shown; as shown in step S42b of FIG. 8, a process of playing back the multiple acquired videos at high speed and using a machine learning model to extract a still image of the face at the timing when it is determined that the face is most shown; and as shown in step S42c of FIG. 8, a process of playing back the multiple acquired videos at high speed and using a machine learning model to extract facial features at the timing when it is determined that the face is most shown and extract a still image of the face.

[0073] In addition, instead of the processing operation in step S43 in Figure 7, as shown in step S43a in Figure 8, a large number of acquired moving images are played back at high speed, still images from the entire moving image are extracted at regular frame intervals, and a machine learning model is used for each extracted still image to extract the still image with the highest probability of depicting a face.

[0074] Thereafter, the many still images extracted in the processing procedure of steps S42a to S42c (processing procedure different from the other steps) and the many still images extracted in the processing procedure of step S43a (processing procedure different from the other steps) are integrated and temporarily stored in a local cache memory. In this manner, in this modified embodiment, still images are extracted from a moving image using four mutually different processing procedures, and the still images extracted by these four processing procedures are integrated to extract still images. Since the final image obtained differs depending on the processing procedures, the reliability is significantly increased compared to when facial images are extracted by only one procedure.

[0075] As described above, in this embodiment, the video crawler of the video crawler unit 17 acquires video data from a WEB site corresponding to a URL stored in the URL list unit 15b, that is, a WEB site including a keyword registered as a cause of face image leakage. That is, the entire Internet is not searched. Therefore, the face image of the user leaked onto the Internet can be detected quickly at an early stage, and the configuration of the image search device is simplified. In addition, in this embodiment, when extracting a plurality of still image data from the acquired video data and acquiring image data including face information from the extracted still image data, the still image data is extracted from the video data using a plurality of different processing procedures, and the still image data extracted by these plurality of different processing procedures is integrated. In this way, since the final image obtained differs depending on the processing procedure, the reliability of face image detection is significantly higher than when the face image is extracted by only one procedure. Therefore, it is possible to reliably detect that the face image of the user has been leaked onto the Internet with a simple configuration.

[0076] In the above-described embodiments and modifications, the image search device of the present invention is constructed using servers, storage, and databases provided by a cloud computing service, but the present invention may also be realized using a dedicated image search device constructed by assembling local servers, storage, and databases without using a cloud computing service.

[0077] The above-described embodiments are merely illustrative of the present invention, and are not limiting, and the present invention can be embodied in various other modified and altered forms. Therefore, the scope of the present invention is defined only by the claims and their equivalents. [Explanation of symbols]

[0078] 10 User terminal 11 Image search server 12. Internet 13 Input / Output and Image Comparison Section 13a Database (AWS / RDS) 13b, 16a, 17a Storage (AWS / S3) 14 Cloud Server 15 URL Crawler Section 15a Cloud Server (AWS / EC2) 15b URL list section 16 Still image crawler section 16b Image search section 16c Machine Learning Department 17 Video Crawler Section 17b Image extraction section

Claims

1. The device includes an input / output and image comparison unit that stores face image data representing a user's own face image registered by the user and a keyword registered by the user as a cause of the face image being leaked, and an image extraction unit that extracts a plurality of still image data from video data acquired from a web site that is a sexual site on the Internet and includes the keyword registered by the user, and acquires image data including face information from the extracted still image data, the input / output and image comparison unit is configured to compare the image data including the face information acquired by the image extraction unit with face image data registered by the user and stored in the input / output and image comparison unit, and to extract a URL of face image data having a high similarity; The image search device is characterized in that the image extraction unit is configured to extract still image data from the video data using a plurality of mutually different processing procedures, and to integrate the still image data extracted by the plurality of mutually different processing procedures.

2. The image search device of claim 1, characterized in that the multiple different processing procedures by the image extraction unit include a process of playing back the video data at high speed and extracting still image data that is determined to contain a face by a machine learning model, and a process of playing back the video data at high speed and extracting still image data at any fixed frame interval.

3. The image search device of claim 1, characterized in that the multiple different processing procedures by the image extraction unit include a process of playing back the video data at high speed and extracting a still image when a machine learning model determines that a face is included, a process of playing back the video data at high speed and extracting a still image of a face at a timing when the machine learning model determines that the face is most included, a process of playing back the video data at high speed and extracting facial features at a timing when the machine learning model determines that the face is most included, and a process of playing back the video data at high speed, extracting still images of the entire video at regular frame intervals, and extracting a still image that has the highest probability of including a face based on the machine learning model for each extracted still image.

4. 2. The image search device according to claim 1, wherein the image extraction unit is configured to compare a confidence value of the image data including the face information with a threshold value to extract image data that most likely shows a face image.

5. 5. The image search device according to claim 4, wherein the image extraction unit is configured to divide the image data including the facial information into a predetermined number of parts from the beginning, select the image data with the highest accuracy among the predetermined number of divided image data, and compare the confidence value of the selected image data with a threshold value.

6. The image search device of claim 1, further comprising a URL crawler unit that collects URLs of web sites that are sexual sites on the Internet and contain keywords registered by the user, and the image extraction unit is configured to obtain video data from the web sites corresponding to the URLs collected by the URL crawler unit.

7. The image search device according to claim 1, characterized in that the input / output and image comparison unit is configured to store facial image data and keywords transmitted from a user terminal, and is configured to transmit the extracted URL to the user terminal.

8. Computer, a program for functioning as an input / output and image comparison means for storing face image data representing a user's own face image registered by the user and a keyword registered by the user as a cause of the face image being leaked, and an image extraction means for extracting a plurality of still image data from video data acquired from a WEB site on the Internet which is a sexual site and includes a keyword registered by the user, and acquiring image data including face information from the extracted still image data, the input / output and image comparison means is means for comparing image data including the acquired face information with face image data registered by the user, and extracting a URL of face image data having a high similarity; The image extraction means extracts still image data from moving image data using a plurality of mutually different processing procedures, and integrates the still image data extracted by the plurality of mutually different processing procedures.

9. Computer, A computer-readable recording medium having recorded thereon a program for functioning as an input / output and image comparison means for storing face image data representing a user's own face image registered by the user and a keyword registered by the user as a cause of the face image being leaked, and an image extraction means for extracting a plurality of still image data from video data acquired from a web site which is a sexual site on the Internet and includes a keyword registered by the user, and acquiring image data including face information from the extracted still image data, the input / output and image comparison means is means for comparing image data including the acquired face information with face image data registered by the user, and extracting a URL of face image data having a high similarity; A computer-readable recording medium, characterized in that the image extraction means is a means for extracting still image data from moving image data using a plurality of mutually different processing procedures and integrating the still image data extracted by the plurality of mutually different processing procedures.

Images