Equipment and methods for photographing subjects

Abstract

[Problem] To provide: a subject photographing device with which an inappropriate image photographed by a user cannot be uploaded to an external network; and a subject photographing method using the same. [Solution] Provided is a subject photographing device comprising: a photographing device 19 capable of photographing a subject; an image photographing / videographing unit 24 that is activated by a user operating a shutter button, and temporarily stores, in a memory, a state of the subject photographed by the photographing device 19; an inappropriate image inference / determination unit 25 that extracts the state of the subject from the memory and determines an inappropriate image by inferring whether or not the state of the subject is the inappropriate image on the basis of inappropriate image model data; an inappropriate image processing unit 26 that does not store the state of the subject in an image storage area of a computer when the determination unit determines that the image of the subject is the inappropriate image; and an appropriate image processing unit 27 that stores the state of the subject in the image storage area of the computer when the determination unit determines that the image of the subject is not an inappropriate image.

Description

Technical Field

[0001] This invention relates to subject photographing devices such as cameras and smartphones with cameras, and methods thereof.

Background Art

[0002] Recently, incidents have frequently occurred where children take inappropriate images with smartphones or the like and share them on SNS, which has become a social problem.

[0003] Especially in the case of camera devices (mainly smartphones, etc.) having a network access function, sharing of captured images and videos is easy, and even those captured by individuals without the intention of spreading them can be generally made public via an image upload server or the like. Such situations have also occurred.

Summary of the Invention

Problems to be Solved by the Invention

[0004] This invention has been made in view of such circumstances, and an object thereof is to provide a subject photographing device and a subject photographing method using the same, in which inappropriate images captured by a user cannot be uploaded onto an external network.

Means for Solving the Problems

[0005] To solve the above problems, according to the first main aspect of this invention, subject photographing apparatuses of the following inventions (1) to (6) are provided.

[0006] (1) An imaging device capable of photographing a subject, An image / video imaging unit that is activated by a user operating a shutter button and temporarily stores the state of the subject imaged by the imaging device in a memory, An inappropriate image inference / determination unit that retrieves the state of the subject from the memory, executes an inference as to whether the state of the subject is an inappropriate image based on inappropriate image model data, and makes a determination of an inappropriate image. If the determination unit determines that the image of the subject is an inappropriate image, the inappropriate image processing unit does not save the state of the subject to the computer's image storage area. If the determination unit does not determine that the image of the subject is an inappropriate image, the appropriate image processing unit saves the state of the subject to the computer's image storage area. A subject photography device characterized by having the following features.

[0007] (2) In the subject photography equipment described in (1) above, The aforementioned image / video capturing unit, in the case of video, sequentially stores the state of the subject at regular intervals from the start of recording into memory. In the case of a video, the aforementioned inappropriate image inference and determination unit sequentially retrieves the state of the subject stored in the memory at regular intervals and performs inference. In the case of a video, the aforementioned inappropriate image processing unit further includes the step of stopping the recording of the video if the inference determines that the state of the subject is an inappropriate image. A subject photography device characterized by the following features.

[0008] (3) In the subject photography equipment described in (1) above, Furthermore, an inappropriate image model data update unit connects to an internet data source to update the aforementioned inappropriate image model data to the latest version. A subject-capturing device characterized by having the following features.

[0009] (4) In the subject photography equipment described in (1) above, moreover, If the aforementioned inference determines that an image is inappropriate, the inappropriate image notification unit notifies a third party of the determination result. A subject-capturing device characterized by having the following features.

[0010] (5) In the subject photography equipment described in (1) above, Furthermore, an inappropriate image scanning unit scans the image and video files stored in the image storage area and performs inference and determination using the inappropriate image inference and determination unit. A subject-capturing device characterized by having the following features.

[0011] (6) In the subject photography equipment described in (5) above, If the aforementioned inappropriate image scanning unit determines, as a result of performing the above inference, that an image stored in the image storage area is inappropriate, it deletes the image or video file from the image storage area. A subject photography device characterized by the following features.

[0012] Furthermore, according to a second main aspect of this invention, the following methods for photographing subjects are provided: (7) to (12).

[0013] (7) A method of photographing a subject using a subject-photography device having an imaging device capable of photographing a subject, The process involves an image / video capture step in which the user operates the shutter button of the camera, causing the camera to temporarily store the state of the subject captured by the imaging device in memory, and The imaging device retrieves the state of the subject from the memory, performs an inference and determination step to determine whether the state of the subject is an inappropriate image based on the inappropriate image model data, and determines that it is an inappropriate image. If the imaging device determines that the image of the subject is an inappropriate image by the determination unit, the device performs an inappropriate image processing step in which it does not save the state of the subject to the computer's image storage area. If the imaging device does not determine that the image of the subject is an inappropriate image by the determination unit, the appropriate image processing step involves saving the state of the subject to the image storage area of ​​the computer. A method for photographing a subject, characterized by having the following features.

[0014] (8) In the subject photography method described in (7) above, In the case of a video, the image / video capturing step sequentially stores the states of the subject at regular intervals from the start of shooting in a memory. In the case of a video, the inappropriate image inference / determination step sequentially retrieves the states of the subject stored in the memory at regular intervals and performs inference. In the case of a video, the inappropriate image processing step further includes a step of stopping the shooting of the video when it is determined that the state of the subject is an inappropriate image as a result of the execution of the inference. A subject shooting method characterized by the above.

[0015] (9) In the subject shooting method according to (7) above, Furthermore, an inappropriate image model data update step in which the subject shooting device connects to a data source on the Internet and updates the inappropriate image model data to the latest one A subject shooting method characterized by having the above.

[0016] (10) In the subject shooting method according to (7) above, Furthermore, an inappropriate image notification step in which the subject shooting device notifies a third party of the determination result when it is determined to be an inappropriate image as a result of the execution of the inference A subject shooting method characterized by having the above.

[0017] (11) In the subject shooting method according to (7) above, Furthermore, an inappropriate image scanning step in which the subject shooting device scans the image / video files stored in the storage area of the image and causes the inappropriate image inference / determination unit to perform inference and determination A subject shooting method characterized by having the above.

[0018] (12) In the subject shooting method according to (11) above, The aforementioned inappropriate image scanning step deletes the image or video file from the image storage area if, as a result of performing the above inference, it is determined that the image stored in the image storage area is inappropriate. A subject photography method characterized by the following features.

[0019] Furthermore, according to a third main aspect of this invention, the following computer software programs of invention (13) to (18) are provided.

[0020] (13) A computer software program for photographing a subject with a subject-photography device having an imaging device capable of photographing a subject, The process involves an image / video capture step in which the camera temporarily stores the state of the subject captured by the imaging device in memory when the user operates the shutter button of the camera, The imaging device retrieves the state of the subject from the memory, performs an inference and determination step to determine whether the state of the subject is an inappropriate image based on the inappropriate image model data, and determines that it is an inappropriate image. If the imaging device determines that the image of the subject is an inappropriate image by the determination unit, the device performs an inappropriate image processing step in which it does not save the state of the subject to the computer's image storage area. If the imaging device does not determine that the image of the subject is an inappropriate image by the determination unit, the appropriate image processing step involves saving the state of the subject to the image storage area of ​​the computer. A computer software program characterized by causing the execution of a certain action.

[0021] (14) In the computer software program described in (13) above, In the case of video, the aforementioned image / video capture process sequentially stores the state of the subject at regular intervals from the start of recording into memory. In the case of video, the aforementioned inappropriate image inference and determination process sequentially retrieves the state of the subject at regular intervals stored in the memory and performs inference. The aforementioned inappropriate image processing step, in the case of a video, further includes a step of stopping the recording of the video if the inference determines that the state of the subject is an inappropriate image. A computer software program characterized by the following:

[0022] (15) In the computer software program described in (13) above, moreover, An inappropriate image model data update process in which the subject-capturing device connects to a data source on the internet and updates the inappropriate image model data to the latest version. A computer software program characterized by having [a certain characteristic].

[0023] (16) In the computer software program described in (13) above, moreover, If the subject-capture device determines, through the execution of the aforementioned inference, to be an inappropriate image, an inappropriate image notification process is performed to notify a third party of the determination result. A computer software program characterized by having [a certain characteristic].

[0024] (17) In the computer software program described in (13) above, moreover, The subject-capturing device scans the image and video files stored in the image storage area, and the inappropriate image inference and determination unit performs inference and determination in the inappropriate image scanning process. A computer software program characterized by having [a certain characteristic].

[0025] (18) In the computer software program described in (17) above, The aforementioned inappropriate image scanning step deletes the image or video file from the image storage area if, as a result of performing the above inference, it is determined that the image stored in the image storage area is inappropriate. A computer software program characterized by the following: [Effects of the Invention]

[0026] According to the configuration of the present invention described above, the camera itself can perform inference and obtain results to determine inappropriate images without using an API implemented on a network, thus preventing inappropriate still images or videos from being stored in the image storage area of ​​the camera.

[0027] Furthermore, features of this invention other than those described above can be understood from the following description of embodiments and drawings. [Brief explanation of the drawing]

[0028] [Figure 1] Figure 1 is an overall configuration diagram showing one embodiment of this invention. [Figure 2] Figure 2 is a schematic diagram showing the imaging equipment. [Modes for carrying out the invention]

[0029] Hereinafter, one embodiment of this invention will be described with reference to the accompanying drawings.

[0030] In Figure 1, reference numeral 1 indicates a smartphone with a camera, which is one embodiment of the imaging device of this invention. In this embodiment, the smartphone 1 is owned by a child and can be monitored and controlled from a parent's smartphone or PC 3 via the internet 2.

[0031] Furthermore, this smartphone 1 can synchronize with a file server 6 on the cloud via the internet 2. This allows the image storage area (camera roll) where images and videos taken with this smartphone 1 are stored to be periodically synchronized with the file server 6, enabling the image and video files stored on this smartphone 1 to be backed up to the cloud.

[0032] Furthermore, as will be explained in more detail later, the smartphone 1 is configured to connect to the inappropriate image model data storage server 7 via the internet 2. This inappropriate image model data storage server 7 has an inappropriate image model data DB 8 that stores inappropriate image model data constructed by machine learning.

[0033] Figure 2 shows the detailed configuration of the camera-equipped smartphone 1 described above.

[0034] This smartphone 1 is actually a portable miniature computer, and a program storage unit 16 and a data storage unit 17 are connected to a bus 15 to which various processing units 10 including a CPU and GPU, RAM 11, input / output IF 12 and communication IF 13 are connected.

[0035] The input / output IF12 is connected to the smartphone's imaging device (image sensor) 19, physical shutter 20, keyboard / touch sensor 21, and display 22. The communication IF13 is connected to the internet 2 via a local network.

[0036] The program storage unit 16 stores, to list only the components relevant to this invention, an OS (not shown), an inappropriate image model data update unit 23, an image / video capture unit 24, an inappropriate image inference / determination unit 25, an inappropriate image processing unit 26, an appropriate image processing unit 27, an inappropriate image notification unit 28, and an inappropriate image scanning unit 29.

[0037] In addition, the data storage unit 17 stores the inappropriate image model data 30, as well as an image storage area 31 and various setting information 32.

[0038] The program storage unit 16 and data storage unit 17 are specifically storage media such as hard disks, memory cards, and SSDs. The functional units 23 to 28 stored in the program storage unit 16 are called as appropriate by the arithmetic unit 10, loaded onto the RAM 11, and executed, thereby functioning as the respective components of this invention.

[0039] The configurations of each of the above-mentioned functional units 23 to 28 will be explained in detail below through their operation.

[0040] First, the inappropriate image model data update unit 23 periodically connects to the inappropriate image model data storage server 7 via the communication IF 13 and the Internet 2. The inappropriate image model data storage server 7 is equipped with an inappropriate image model data DB 8 that stores inappropriate model data generated by machine learning. The inappropriate image model data update unit 23 accesses this inappropriate image model data DB 8 to download the updated model data 30 and store it in the data storage unit 17. As a result, the model data 30 stored in the smartphone 1 is always kept up-to-date.

[0041] Next, the user of the smartphone 1 activates the image / video capturing unit 24 on the smartphone 1. This activation can be done, for example, by clicking the camera icon displayed on the smartphone 1's display 22, or it may be set to activate automatically when the smartphone 1 is turned on.

[0042] Next, when the user points the lens of the imaging device 19 of the smartphone 1 at a subject and operates the shutter 20, the image / video capturing unit 24 temporarily stores the state of the subject at that moment in the RAM 11. This shutter may be the physical shutter 20, or it may be a software shutter displayed on the display by the image / video capturing unit 24.

[0043] Then, the inappropriate image inference and determination unit 25 retrieves the state of the subject temporarily stored in the RAM 11, performs inference based on the inappropriate image model data 30 stored in the data storage unit 17, and determines whether the state of the subject belongs to an inappropriate image. This determination is made, for example, by determining whether the response value obtained from the inference is above a certain value.

[0044] Specifically, in this embodiment, the inappropriate image inference and judgment unit 25 matches the state of the subject with the model data 30 and calculates a degree of agreement (0 to 1), and the judgment is, for example, 0.8 or higher. In this embodiment, a CNN (Convolutional Neural Network) is used as the inference algorithm, but other algorithms may also be used.

[0045] Furthermore, when recording a subject in video, the inappropriate image inference and determination unit 25 retrieves the subject's state from the RAM 11 at regular intervals (approximately every second) from the start of recording, and continuously performs the above inference and determination.

[0046] If an inappropriate image is determined as described above, the inappropriate image processing unit 26 will delete the still image from the RAM 11 without saving it to the image storage area 31. In the case of a video, the inappropriate image processing unit 26 will stop recording the video.

[0047] On the other hand, if it is determined that the image is not inappropriate, the appropriate image processing unit 27 stores the still image in the image storage area 31, and continues recording the video, and after completion, stores it in the image storage area 31 as well.

[0048] In other words, in either case, the system prevents images and videos from being saved to the smartphone 1 by determining if they are inappropriate images before they are saved to the image storage area 31 accessible to the user.

[0049] Furthermore, the inappropriate image notification unit 28 displays a notification on the display 22 indicating that an inappropriate image has been taken. The inappropriate image notification unit 28 can also send notifications to external parties by referring to notification destination information in the various information 32 stored in the data storage unit. In this embodiment, the system is configured to notify the parent's smartphone 3, which supervises the user of this smartphone 1, via email that an inappropriate image has been taken.

[0050] In this embodiment, as an option, the system includes an inappropriate image scanning unit 29 that periodically checks images and video data stored in the image storage area 31, even if they were not taken by the user, by downloading them, and sends them to the inappropriate image inference / determination unit 25 for determination of inappropriate images. If an image or video is determined to be inappropriate, the scanning unit 29 can delete it.

[0051] Whether or not to use this scanning unit 29 can be specified by registering it in the various setting information 32.

[0052] Although one embodiment of the present invention has been described above, it goes without saying that the present invention is not limited to the above embodiment and can be modified in various ways without changing the gist of the invention. [Explanation of Symbols]

[0053] 1…Smartphone with camera 2…Internet 3… Smartphone 6…File Server 7…Server for storing inappropriate image model data 8…Inappropriate image model data database 10...Arithmetic device 11…RAM 12…Input / Output Interface 13…Communication IF 15... Bus 16…Program storage unit 17...Data storage unit 18...Data storage unit 19…Imaging device 20…Physical shutter 21... Touch sensor 22…Display 23... Inappropriate Image Model Data Update Section 24…Image and video capture unit 25…Inappropriate image inference / determination unit 26…Inappropriate Image Processing Unit 27…Appropriate Image Processing Unit 28... Inappropriate Image Notification Department 29... Inappropriate Image Scanning Section 30…Inappropriate image model data 31...Image storage area 32… Various settings information

Claims

1. An imaging device capable of photographing a subject, An image / video capturing unit that is activated when the user operates the shutter button and temporarily stores the state of the subject captured by the imaging device in memory, An inappropriate image inference and determination unit retrieves the state of the subject from the memory, performs inference on whether the state of the subject is an inappropriate image based on inappropriate image model data constructed by machine learning obtained from data sources on the internet, and determines whether it is an inappropriate image. If the determination unit determines that the image of the subject is an inappropriate image, the inappropriate image processing unit does not save the image of the subject to the computer's image storage area. If the determination unit does not determine that the image of the subject is an inappropriate image, the appropriate image processing unit saves the image of the subject to the computer's image storage area. An inappropriate image model data update unit connects to the aforementioned data source on the Internet and updates the inappropriate image model data obtained from the aforementioned data source on the Internet to the latest version. A subject photography device characterized by having the following features.

2. In the subject photography device described in claim 1, The aforementioned image / video capturing unit, in the case of video, sequentially stores the state of the subject at regular intervals from the start of recording into memory. In the case of a video, the aforementioned inappropriate image inference and determination unit sequentially retrieves the state of the subject stored in the memory at regular intervals and performs inference. In the case of a video, the aforementioned inappropriate image processing unit further includes the step of stopping the recording of the video if the inference determines that the state of the subject is an inappropriate image. A subject photography device characterized by the following features.

3. In the subject photography device according to claim 1, moreover, If the aforementioned inference determines that an image is inappropriate, the inappropriate image notification unit notifies a third party of the determination result. A subject-capturing device characterized by having the following features.

4. In the subject photography device according to claim 1, Furthermore, an inappropriate image scanning unit scans the image and video files stored in the image storage area and performs inference and determination using the inappropriate image inference and determination unit. A subject-photography device characterized by possessing the ability to capture images.

5. In the subject photography device according to claim 4, If the aforementioned inappropriate image scanning unit determines, as a result of performing the above inference, that an image stored in the image storage area is inappropriate, it deletes the image or video file from the image storage area. A subject photography device characterized by the following features.

6. A method of photographing a subject using a subject-photography device having an imaging device capable of photographing a subject, The process involves an image / video capture step in which the user operates the shutter button of the camera, causing the camera to temporarily store the state of the subject captured by the imaging device in memory, and The imaging device retrieves the state of the subject from the memory, performs an inappropriate image inference and determination step, which involves inferring whether the state of the subject is an inappropriate image based on inappropriate image model data constructed by machine learning obtained from data sources on the internet, and determining whether it is an inappropriate image. If the imaging device determines that the image of the subject is an inappropriate image by the determination step, it performs an inappropriate image processing step in which it does not save the image of the subject to the computer's image storage area. If the imaging device does not determine that the image of the subject is an inappropriate image by the determination step, the appropriate image processing step saves the image of the subject to the computer's image storage area. An inappropriate image model data update step, which involves connecting to the aforementioned data source on the Internet and updating the inappropriate image model data obtained from the aforementioned data source on the Internet to the latest version. A method for photographing a subject, characterized by having the following features.

7. In the subject photography method according to claim 6, In the case of video, the aforementioned image / video capture process sequentially stores the state of the subject at regular intervals from the start of recording into memory. In the case of video, the aforementioned inappropriate image inference and determination process sequentially retrieves the state of the subject at regular intervals stored in the memory and performs inference. The aforementioned inappropriate image processing step, in the case of a video, further includes a step of stopping the recording of the video if the inference determines that the state of the subject is an inappropriate image. A subject photography method characterized by the following features.

8. In the subject photography method according to claim 6, moreover, If the subject-capture device determines, through the execution of the aforementioned inference, to be an inappropriate image, an inappropriate image notification process is performed to notify a third party of the determination result. A method for photographing a subject, characterized by having the following features.

9. In the subject photography method according to claim 6, moreover, The subject-capture device scans the image and video files stored in the image storage area, and the inappropriate image inference and determination unit performs inference and determination in the inappropriate image scanning process. A method for photographing a subject, characterized by possessing a certain feature.

10. In the subject photography method according to claim 9, The aforementioned inappropriate image scanning process deletes the image or video file from the image storage area if, as a result of the inference, it is determined that the image stored in the image storage area is inappropriate. A subject photography method characterized by the following features.

11. A computer-readable storage medium containing a computer software program for photographing a subject using a subject-photography device having an imaging device capable of photographing a subject, The process involves an image / video capture step in which the camera temporarily stores the state of the subject captured by the imaging device in memory when the user operates the shutter button of the camera, The imaging device retrieves the state of the subject from the memory, performs an inappropriate image inference and determination step, which involves inferring whether the state of the subject is an inappropriate image based on inappropriate image model data constructed by machine learning obtained from data sources on the internet, and determining whether it is an inappropriate image. If the imaging device determines that the image of the subject is an inappropriate image by the determination step, it performs an inappropriate image processing step in which it does not save the image of the subject to the computer's image storage area. If the imaging device does not determine that the image of the subject is an inappropriate image by the determination step, the appropriate image processing step saves the image of the subject to the computer's image storage area. An inappropriate image model data update step, which involves connecting to the aforementioned data source on the Internet and updating the inappropriate image model data obtained from the aforementioned data source on the Internet to the latest version. A computer-readable storage medium containing a computer software program characterized by its execution.

12. A computer-readable storage medium recording the computer software program described in claim 11, In the case of video, the aforementioned image / video capture process sequentially stores the state of the subject at regular intervals from the start of recording into memory. In the case of video, the aforementioned inappropriate image inference and determination process sequentially retrieves the state of the subject at regular intervals stored in the memory and performs inference. The aforementioned inappropriate image processing step, in the case of a video, further includes a step of stopping the recording of the video if the inference determines that the state of the subject is an inappropriate image. A computer-readable storage medium on which a computer software program is recorded, characterized by the following features.

13. A computer-readable storage medium recording a computer software program according to claim 11, further comprising: If the subject-capture device determines, through the execution of the aforementioned inference, to be an inappropriate image, an inappropriate image notification process is performed to notify a third party of the determination result. A computer-readable storage medium on which a computer software program is recorded is characterized by having the following:

14. A computer-readable storage medium recording a computer software program according to claim 11, further comprising: The subject-capture device scans the image and video files stored in the image storage area, and the inappropriate image inference and determination unit performs inference and determination in the inappropriate image scanning process. A computer-readable storage medium on which a computer software program is recorded, characterized by possessing the following properties.

15. A computer-readable storage medium recording the computer software program described in claim 14, The aforementioned inappropriate image scanning process deletes the image or video file from the image storage area if, as a result of the inference, it is determined that the image stored in the image storage area is inappropriate. A computer-readable storage medium on which a computer software program is recorded, characterized by the following features.

Images