Information processing system

Abstract

To provide an information processing system that allows processing with privacy taken into consideration.SOLUTION: An information processing system 1 includes a plurality of illumination devices 100, a plurality of repeaters 200, and a plurality of server devices 300, and a plurality of display devices 400. The information processing system includes a processing unit for performing conversion processing of converting first information indicating personal information of a person TG in an image taken by a camera unit 20 of the illumination devices 100 to second information indicating attribute information of the person, in the repeaters 200, the server devices 300, and the illumination devices.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing system.

Background Art

[0002] Conventionally, processing using an image captured by a camera or the like has been performed. For example, a technique has been proposed in which imaging is performed by a camera or the like provided in a lighting device, and information is managed using the captured image (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there is room for improvement in the above conventional technology. For example, in the conventional technology, when a person is imaged and the image includes a person, this has not been considered. Thus, there is room for improvement in the conventional technology from the perspective of human privacy.

[0005] An object of the present invention is to provide an information processing system that enables processing considering privacy.

Means for Solving the Problems

[0006] The information processing system of the present embodiment includes a processing unit that executes a conversion process for converting first information indicating personal information of a person included in an image captured by a camera into second information indicating attribute information of the person.

Effects of the Invention

[0007] According to the present invention, it is possible to enable processing considering privacy. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an information processing system according to an embodiment. [Figure 2] Figure 2 is a perspective view showing an example of a lighting device according to this embodiment. [Figure 3] Figure 3 is a block diagram showing the configuration of a lighting device according to an embodiment. [Figure 4] Figure 4 is a block diagram showing the configuration of a repeater according to this embodiment. [Figure 5] Figure 5 is a block diagram showing the configuration of a server device according to this embodiment. [Figure 6] Figure 6 shows an example of masking processing performed by an information processing system. [Figure 7] Figure 7 shows an example of masking based on the quantification of facial features. [Figure 8] Figure 8 shows an example of data transformation. [Figure 9] Figure 9 is a flowchart showing an example of data conversion processing by an information processing system. [Figure 10] Figure 10 shows an example of the components related to filtering processing. [Figure 11] Figure 11 shows an example of attribute data. [Figure 12] Figure 12 is a flowchart showing an example of filtering based on risk level. [Figure 13] Figure 13 is a flowchart showing an example of filtering based on risk content. [Modes for carrying out the invention]

[0009] The information processing system 1 according to the embodiment described below has a processing unit (for example, processing unit 332, processing unit 232, or processing unit 42) that executes a conversion process of converting first information indicating personal information of a person included in an image captured by camera 21 into second information indicating the person's attribute information.

[0010] Also, in the information processing system 1 according to the embodiment described below, the processing unit executes a conversion process of converting first information including at least one of face information indicating a person's face and identification information for identifying the person into second information including at least one of the person's age and gender.

[0011] Also, in the information processing system 1 according to the embodiment described below, the processing unit executes a conversion process of converting first information including numerical values obtained by converting a person's face included in an image for each part including at least one of eyes, nose, and mouth into second information.

[0012] Also, in the information processing system 1 according to the embodiment described below, the processing unit generates additional information including at least one of interest information indicating a person's interests and movement information indicating the person's movement direction.

[0013] Also, the information processing system 1 according to the embodiment described below includes an edge (for example, lighting device 100) including a light source 31 and a camera 21, and an information processing device (for example, server device 300, repeater 200, or lighting device 100) including a processing unit. The information processing device executes a conversion process of converting first information of a person included in an image acquired from the edge into second information of the person.

[0014] [Embodiment] [Configuration of Information Processing System] Hereinafter, the device configuration of the information processing system 1 according to the embodiment and the functions of each device will be described. First, the device configuration of the information processing system 1 according to the embodiment will be described based on FIG. 1. FIG. 1 is a diagram showing the information processing system according to the embodiment.

[0015] The information processing system 1 shown in FIG. 1 includes a lighting device 100, a repeater 200, a server device 300, and a display device 400. The information processing system 1 may include a plurality of lighting devices 100, a plurality of repeaters 200, a plurality of server devices 300, and a plurality of display devices 400. The lighting device 100 is communicably connected to the repeater 200 via a LAN (Local Area Network) or the like, either wired or wirelessly. The lighting device 100 communicates with the server device 300 via the repeater 200.

[0016] The server device 300 is communicably connected to the repeater 200 and the display device 400 via a predetermined communication network (network) such as the Internet, either wired or wirelessly. Note that the server device 300 may be connected to the repeater 200 or the display device 400 in any manner as long as information can be transmitted and received, and may be communicably connected either wired or wirelessly. For example, the server device 300 may communicate with the display device 400 via an arbitrary device (such as a gateway device).

[0017] The lighting device 100 is a lighting device with a camera. For example, the lighting device 100 is a device in which the configuration of the camera is integrated with the lighting fixture. The lighting device 100 is an edge that includes a light source 31 and a camera 21. For example, the lighting device 100 is an information processing device (computer) that executes processing related to the protection of personal information. When a person's face is included in an image, the lighting device 100 executes a masking process if the person's face included in the image satisfies the masking condition. Note that the masking process may be performed by the repeater 200 or the server device 300.

[0018] For example, the lighting device 100 is a lighting device installed on the ceiling of a structure such as a building, which illuminates the floor surface when lit. Here, the lighting device 100 will be described with reference to Figure 2. Figure 2 is a perspective view showing an example of a lighting device according to the embodiment. The lighting device 100 according to the embodiment is a ceiling-mounted type lighting device, a so-called base light. In the example shown in Figure 2, the lighting device 100 has a camera unit 20 and a lighting unit 30. Note that the lighting device 100 is not limited to a base light, but may also be a ceiling light, downlight, spotlight, etc.

[0019] The camera unit 20 is a shooting device that captures a predetermined area around the lighting device 100. In Figure 1, the camera unit 20 captures the area AR corresponding to the field of view of the camera 21. For example, the area AR indicates the shooting range of the camera 21. For example, if a person TG is located within the area AR, the camera unit 20 captures an image (video) including the person TG. The lighting unit 30 is a lighting device that illuminates at least a part of the shooting area of ​​the camera unit 20 by turning on a light source. That is, the shooting area of ​​the camera unit 20 and the area illuminated by the lighting unit 30 may or may not coincide. Also, the camera unit 20 does not have to be disposed integrally with the lighting unit 30, and may be disposed with a space between the camera unit 20 and the lighting unit 30.

[0020] The repeater 200 is a device that has the function of performing relay processing. For example, the repeater 200 is an information processing device (computer) that functions as a so-called gateway. For example, the repeater 200 performs processing related to the protection of personal information. The repeater 200 performs a conversion process that converts the first information of a person contained in the image acquired from the edge device such as the lighting device 100 into second information of a person. Note that the conversion process may be performed by the edge device such as the lighting device 100 or the server device 300.

[0021] The server device 300 is a device used as a server in a client-server system. For example, the server device 300 is an information processing device (computer) that functions as a so-called cloud server. For example, the server device 300 performs processing related to the protection of personal information. The server device 300 performs filtering processing on the image according to the results of detecting a person and analyzing the person's surroundings in the image acquired from the camera 21. Note that the filtering processing may be performed by edge devices such as the lighting device 100 or the repeater 200. The server device 300 performs processing on the image according to the specifications regarding the protection of personal information received for the image acquired from the camera 21.

[0022] The display device 400 is a device that has the function of displaying information. For example, the display device 400 is an information processing device (computer) used by a user. For example, the display device 400 is a portable terminal device that can communicate with any server device 300 via a predetermined network. Figure 1 shows the case where the display device 400 is a notebook PC (Personal Computer). Note that the display device 400 may be a smart device such as a smartphone or tablet, or a computer such as a desktop PC, in addition to a notebook PC.

[0023] The display device 400 displays information provided by the server device 300. For example, the display device 400 sends information requesting information (request information) to the server device 300 and displays the information received from the server device 300. For example, the display device 400 sends request information specifying information to the server device 300 and displays the information received from the server device 300. For example, the display device 400 displays data that has undergone processing related to the protection of personal information (processed data).

[0024] [Example of lighting system configuration] Next, the configuration of the lighting device 100 according to the embodiment will be described using Figure 3. Figure 3 is a block diagram showing the configuration of the lighting device according to the embodiment. For example, the lighting device 100 is an information processing device (computer) that functions as a lighting device with a camera. As shown in Figure 3, the lighting device 100 has a communication unit 10, a camera unit 20, a lighting unit 30, a control unit 40, and a storage unit 50.

[0025] The communication unit 10 transmits and receives information with other devices such as the repeater 200. The communication unit 10 is implemented, for example, by a predetermined communication circuit or a NIC (Network Interface Card). The communication method used by the communication unit 10 may be wireless communication such as a local area network (LAN). The communication method used by the communication unit 10 may be any communication method, such as wired communication or infrared communication.

[0026] The camera unit 20 includes a camera 21, a camera control unit 22, and a storage unit 23. The camera 21 is equipped with an image sensor that electronically acquires images, such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and generates image data (also called "images") by capturing a predetermined angle of view. The camera 21 is positioned in a predetermined location and images a specific space. The image data may be video data or still image data.

[0027] The camera control unit 22 controls the shooting by the camera 21. The camera control unit 22 also outputs the image data generated by the shooting by the camera 21 to the control unit 40.

[0028] The memory unit 23 stores programs and data for implementing various controls of the camera unit 20. The memory unit 23 may also store images captured by the camera 21 or image data generated by the camera 21. The memory unit 23 is implemented using, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or storage devices such as hard disks or optical discs.

[0029] The lighting unit 30 includes a light source 31, a lighting control unit 32, and a storage unit 33. The light source 31 has, for example, a semiconductor light-emitting element such as an LED (Light Emitting Diode). The light source 31 turns on or off in a manner corresponding to a control signal output from the lighting control unit 32.

[0030] The lighting control unit 32 controls the on and off of the light sources 31. The lighting control unit 32 turns on light sources 31 that are to be turned off, or turns off light sources 31 that are to be turned on. In the case of a lighting unit 30 in which the dimming of the light sources 31 is possible, the lighting control unit 32 may also control the dimming degree of the light sources 31.

[0031] The memory unit 33 stores programs and data for realizing various controls of the lighting unit 30. The memory unit 33 is implemented using, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or storage devices such as hard disks or optical discs. The memory unit 33 may also be provided in the memory unit 50, which will be described later.

[0032] The memory unit 50 stores various types of information related to information processing. The memory unit 50 stores programs for implementing various controls of the control unit 40. The memory unit 50 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or storage devices such as hard disks and optical discs. The memory unit 50 stores various types of information used to perform privacy protection processing. The memory unit 50 stores information generated by privacy protection. For example, the memory unit 50 stores information used for masking processing. For example, the memory unit 50 stores information used for skeleton detection. Note that the information held by the memory unit 50 is not limited to the above, and the memory unit 50 may store various types of information depending on the purpose. For example, the memory unit 50 may store information stored in the memory unit 23 or information stored in the memory unit 33.

[0033] The control unit 40 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in internal memory using RAM as a working area. Alternatively, the control unit 40 can be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The control unit 40 has an acquisition unit 41, a processing unit 42, and a transmission unit 43, and implements or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 40 is not limited to the configuration shown in Figure 3, and other configurations are also acceptable as long as they perform the information processing described later.

[0034] The acquisition unit 41 acquires various information. The acquisition unit 41 acquires various information from the camera unit 20. The acquisition unit 41 acquires images captured by the camera unit 20 from the camera unit 20. The acquisition unit 41 acquires various information from the lighting unit 30. The acquisition unit 41 acquires various information from the storage unit 50. The acquisition unit 41 acquires various information from an external information processing device. The acquisition unit 41 receives various information from an external information processing device via the communication unit 10. The acquisition unit 41 receives information from the repeater 200.

[0035] The processing unit 42 processes the image captured by the camera unit 20 and performs predetermined processing. The processing unit 42 performs image processing. The processing unit 42 stores various information in the storage unit 50.

[0036] When the image captured by the camera 21 includes a human face, the processing unit 42 performs a masking process to mask the human face included in the image if the human face included in the image satisfies the masking conditions. The processing unit 42 converts the human face included in the image into numerical values ​​for each part, including at least one of the eyes, nose, and mouth, and performs the masking process if the numerical values ​​satisfy the masking conditions. The processing unit 42 also converts the human face included in the image into numerical values ​​for each part if the angle corresponding to the orientation of the human face falls within a predetermined range, and performs the masking process if the numerical values ​​satisfy the masking conditions.

[0037] The processing unit 42 converts the human face in the image into numerical values ​​for each part if the angle of the human face included in the shooting range of the camera 21 mounted on the ceiling falls within a predetermined range. The processing unit 42 converts the human face in the image into numerical values ​​for each part if the angle between the human face included in the shooting range of the camera 21 mounted on the ceiling and the camera 21 falls within a predetermined range. Based on the detection result of the human face included in the image by skeletal detection, the processing unit 42 executes a masking process if the human face included in the image satisfies the masking conditions.

[0038] The transmitting unit 43 transmits various information to an external device via the communication unit 10. The transmitting unit 43 transmits various information to the repeater 200. The transmitting unit 43 transmits the masked image to the repeater 200.

[0039] [Example of a repeater configuration] Next, the configuration of the repeater 200 according to the embodiment will be described using Figure 4. Figure 4 is a block diagram showing the configuration of the repeater according to the embodiment. As shown in Figure 4, the repeater 200 has a communication unit 210, a storage unit 220, and a control unit 230.

[0040] The communication unit 210 is implemented, for example, by a predetermined communication circuit or NIC. For example, the communication unit 210 can communicate with external devices such as the lighting device 100 and the server device 300. The communication unit 210 may also be able to communicate with any external device, such as a terminal device used by the administrator of the information processing system 1.

[0041] The storage unit 220 is implemented by, for example, a semiconductor memory element such as RAM or flash memory, or a storage device such as a hard disk or optical disc. The storage unit 220 stores various information related to information processing. The storage unit 220 stores information related to the repeater 200. As information related to the repeater 200, the storage unit 220 stores information such as the device ID and installation location. The storage unit 220 stores various information used to perform processing related to privacy protection. The storage unit 220 stores information generated by privacy protection. For example, the storage unit 220 stores information used in data conversion processing. For example, the storage unit 220 stores information such as attribute data generated by data conversion. Note that the information held by the storage unit 220 is not limited to the above, and the storage unit 220 may store various types of information depending on the purpose.

[0042] The control unit 230 is implemented, for example, by a CPU or MPU executing various programs stored in internal memory using RAM as a working area. Alternatively, the control unit 230 can be implemented by an integrated circuit such as an ASIC or FPGA. The control unit 230 has an acquisition unit 231, a processing unit 232, and a transmission unit 233, and implements or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 230 is not limited to the configuration shown in Figure 4, and other configurations are also acceptable as long as they perform the information processing described later.

[0043] The acquisition unit 231 acquires various types of information. The acquisition unit 231 acquires various types of information from the storage unit 220. The acquisition unit 231 receives various types of information from external devices via the communication unit 210. The acquisition unit 231 receives information from the lighting device 100. For example, the acquisition unit 231 receives information collected by the lighting device 100 from the lighting device 100. The acquisition unit 231 may also receive information from the server device 300.

[0044] The processing unit 232 performs various processes using the image. The processing unit 232 performs image recognition processing. The processing unit 232 uses image recognition technology to perform image recognition on the image and generates image recognition data that shows the result of the image recognition. For example, the processing unit 232 recognizes people included in the image. The processing unit 232 determines whether or not the image contains a human face. The processing unit 232 stores various information in the storage unit 220.

[0045] The processing unit 232 performs a conversion process to convert first information, which indicates a person's personal information contained in the image captured by the camera 21, into second information, which indicates a person's attribute information. The processing unit 232 also performs a conversion process to convert first information, which includes at least one of a person's face information and identification information that identifies a person, into second information, which includes at least one of a person's age and gender.

[0046] The processing unit 232 performs a conversion process to convert first information, which includes numerical values ​​converted for each part of a person's face in an image, including at least one of the eyes, nose, and mouth, into second information. The processing unit 232 generates additional information, which includes at least one of interest information indicating the person's interests and movement information indicating the direction of the person's movement.

[0047] The transmitting unit 233 transmits various information to an external device via the communication unit 210. The transmitting unit 233 transmits various information to the lighting device 100 or the server device 300. The transmitting unit 233 transmits the converted information to the server device 300. The transmitting unit 233 transmits additional information to the server device 300.

[0048] [Example of server device configuration] Next, the configuration of the server device 300 according to the embodiment will be described using Figure 5. Figure 5 is a block diagram showing the configuration of the server device according to the embodiment. As shown in Figure 5, the server device 300 has a communication unit 310, a storage unit 320, and a control unit 330.

[0049] The communication unit 310 is implemented, for example, by a predetermined communication circuit or NIC. For example, the communication unit 310 can communicate with external devices such as a repeater 200 or a display device 400. The communication unit 310 may also be able to communicate with any external device, such as a terminal device used by the administrator of the information processing system 1.

[0050] The storage unit 320 is implemented by, for example, a semiconductor memory element such as RAM or flash memory, or a storage device such as a hard disk or optical disc. The storage unit 320 stores various information related to information processing. The storage unit 320 stores various information received from the repeater 200. The storage unit 320 stores various information used to perform privacy protection processing. The storage unit 320 stores information generated by privacy protection. For example, the storage unit 320 stores information used for filtering processing. For example, the storage unit 320 stores images that have undergone privacy protection processing. Note that the information held by the storage unit 320 is not limited to the above, and the storage unit 320 may store various types of information depending on the purpose.

[0051] The control unit 330 is implemented, for example, by a CPU or MPU executing various programs stored in internal memory using RAM as a working area. Alternatively, the control unit 330 can be implemented by an integrated circuit such as an ASIC or FPGA. The control unit 330 has an acquisition unit 331, a processing unit 332, and a transmission unit 333, and implements or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 330 is not limited to the configuration shown in Figure 5, and other configurations are also acceptable as long as they perform the information processing described later.

[0052] The acquisition unit 331 acquires various types of information. The acquisition unit 331 acquires various types of information from the storage unit 320. The acquisition unit 331 receives various types of information from external devices via the communication unit 310. The acquisition unit 331 receives information from the repeater 200. For example, the acquisition unit 331 may receive information collected by the lighting device 100 from the repeater 200.

[0053] The processing unit 332 performs various processes using the image. The processing unit 332 performs image recognition processing. The processing unit 332 uses image recognition technology to perform image recognition on the image and generates image recognition data that shows the result of the image recognition. For example, the processing unit 332 recognizes people included in the image. The processing unit 332 determines whether or not the image contains a person's face. The processing unit 332 stores various information in the storage unit 320.

[0054] If the image captured by the camera 21 contains a person, the processing unit 332 performs a filtering process to apply a predetermined filter to the image, based on the results of detecting the person in the image and analyzing the area around the person. The processing unit 332 determines the filtering to be applied to the image according to the risk level based on the analysis results, and performs a filtering process to apply the determined filtering. The processing unit 332 calculates a larger risk level value the higher the risk of identifying the person. If the color of the person's outline affects the identification of the person, the processing unit 332 performs a filtering process to correct the color of the person's outline.

[0055] The processing unit 332 performs processing on the image in accordance with the specifications regarding the protection of personal information received for the image captured by the camera 21. Depending on the specifications, which include a mode for whether or not to perform processing related to the protection of personal information, the processing unit 332 performs processing related to the protection of personal information on the image if the mode is a protection mode for performing processing related to the protection of personal information.

[0056] The processing unit 332 performs processing related to the protection of personal information on the area of ​​the image, in accordance with the designation that includes the area to be protected for personal information. The processing unit 332 performs processing related to the protection of personal information on the person in the image, in accordance with the designation that includes the person to be protected for personal information. The processing unit 332 performs processing related to the protection of personal information on the person in the image that corresponds to the state, in accordance with the designation that includes at least one of the person's attributes, movement, and appearance.

[0057] The transmitting unit 333 transmits various information to an external device via the communication unit 310. The transmitting unit 333 transmits various information to the display device 400 or the repeater 200. The transmitting unit 333 transmits information that the display device 400 will display to the display device 400. The transmitting unit 333 transmits information that has been processed to protect personal information to the display device 400.

[0058] [Examples of privacy protection processing (Masking examples)] Information processing system 1 performs masking as a privacy protection process. For example, information processing system 1 is a camera-equipped lighting system, and performs personal information protection processing on video data at the edge side of the lighting device 100, etc. This point will be explained below, including the prerequisites. Note that explanations of points similar to those mentioned above will be omitted as appropriate.

[0059] Conventionally, cameras installed in stores and other locations shoot from an oblique angle, which means that when people are captured in the frame, personal information such as faces may be included. In such cases, it becomes necessary to mask the area where faces are visible to protect personal information. Furthermore, because faces are frequently captured when shooting from an oblique angle, masking is required for each video. On the other hand, in information processing system 1, the camera is integrated with the lighting fixture, such as the lighting device 100, and shoots directly downwards from the ceiling, so faces are captured less frequently than before, reducing the effort required for masking.

[0060] Furthermore, the information processing system 1 generates various information (result information) about people included in an image, such as their body parts, movements, face orientation, and facial features, through skeletal detection (skeletal recognition) of the image. The information processing system 1 may also perform image-based skeletal detection processing using a skeletal detection model trained with machine learning technologies such as AI (artificial intelligence). For example, the information processing system 1 takes an image as input and performs image-based skeletal detection processing using a skeletal detection model that outputs information (result information) indicating the skeletal detection results of people included in the input image. Note that any conventional technology can be used for skeletal detection in the information processing system 1 as long as the desired information can be obtained, and the information processing system 1 performs skeletal detection processing using conventional technologies related to skeletal detection as appropriate. For example, when a person (e.g., person TG in Figure 1) is captured in the field of view of the camera 21 (e.g., area AR in Figure 1), the information processing system 1 detects the person's skeleton and determines that a face may be captured when the elevation angle exceeds a certain value. If the system determines that a face may be visible, it performs a masking process to mask the person's face. For example, the system performs a masking process to mask the person's face using a camera-side device such as a lighting device 100.

[0061] This reduces the possibility of identifying individuals and lowers the risk of violating the Personal Information Protection Act. Therefore, information processing system 1 becomes unable to identify individuals, and the effect of protecting personal information is achieved. As described above, information processing system 1 sends captured data to a data server such as a server device 300, which stores and processes the images captured by the lighting device 100, which is installed on the ceiling of a factory or warehouse. When a person enters the field of view of camera 21, information processing system 1 detects the person's skeleton and recognizes them as a person. For example, if a person's face is captured, information processing system 1 converts facial features such as eyes, nose, and mouth into numerical data from the video data, and if the threshold is exceeded, it masks the face of the person in the video. Note that the facial features are not limited to eyes, nose, and mouth; any facial feature such as eyebrows or contours may be used. Furthermore, information processing system 1 sets a threshold within the camera's field of view to check whether the face is visible from the angle of gaze when looking upwards relative to the horizontal direction, and if the threshold is exceeded, it masks the face. For example, the information processing system 1 recognizes the movement of the subject's neck or head within the field of view of the camera 21 using skeletal detection, and performs masking when it detects movement that would include the face.

[0062] Here, an example of masking processing will be explained using Figures 6 and 7. Figure 6 is a diagram showing an example of masking processing by an information processing system. Figure 7 is a diagram showing an example of masking processing based on the digitization of facial features. For example, the lighting device 100 performs the masking processing shown in Figure 6.

[0063] Furthermore, the processing entity performing the masking is not limited to the lighting device 100; other devices included in the information processing system 1, such as the server device 300 or the repeater 200, may also perform the masking. For example, the devices included in the information processing system 1 may share the responsibility for each of the masking processes. For instance, in the information processing system 1, the lighting device 100 or the repeater 200 may perform the digitization of facial features, while the server device 300 or the repeater 200 may perform the masking of the person's face. Thus, any configuration is possible for the devices in the information processing system 1 that perform each of the masking processes.

[0064] As shown in Figure 6, the information processing system 1 performs masking according to the angle at which a person's face is visible. Person TG1 in Figure 6 shows the case where the direction (angle) of the face is facing horizontally, angle AG1. Person TG2 in Figure 6 shows the case where the direction (angle) of the face is facing upwards, angle AG2. In Figure 6, the case where a person moves from the position of person TG1 to the position of person TG2 in the region AR in Figure 6, which corresponds to the field of view of the camera 21 of the lighting device 100.

[0065] If a person is included in the AR region, as shown by person TG2 in Figure 6, the information processing system 1 determines whether to apply masking to that person's face. For example, if a person is included in the AR region, the information processing system 1 converts the person's face in the image into numerical values ​​for each part, including at least one of the eyes, nose, and mouth. If the numerical values ​​satisfy the masking conditions, the system performs the masking process. For example, the information processing system 1 compares the numerical values ​​of each part of the person's face with a threshold value. If the numerical value of each part is greater than or equal to the threshold value, the system determines that the masking conditions are met and performs the masking process.

[0066] The processing example PS1 in Figure 7 shows an example of masking when a person is looking upwards. For example, processing example PS1 shows processing applied to an image taken with person TG2 in Figure 6. First, the information processing system 1 converts the upward-looking face of the person in the image taken with person TG2 into numerical values ​​for each part: eyes, nose, and mouth.

[0067] In Figure 7, the information processing system 1 converts the eyes of a person in an image taken in the state of human TG2 to the numerical value "70". The information processing system 1 also converts the nose of a person in an image taken in the state of human TG2 to the numerical value "60". The information processing system 1 also converts the mouth of a person in an image taken in the state of human TG2 to the numerical value "70".

[0068] The numerical values ​​for each part shown in Figure 7 indicate how clearly they are visible. For example, the numerical values ​​for each part range from 0 to 100, with values ​​closer to 100 indicating that the part is clearly visible.

[0069] For example, information processing system 1 compares the numerical values ​​of each part of a person's face with a predetermined threshold (for example, 50), and if the numerical values ​​of each part are greater than or equal to the threshold, it considers the masking condition to be met and executes the masking process. In Figure 7, information processing system 1 considers the masking condition to be met and executes the masking process if the numerical values ​​of the three parts of a person's face—eyes, nose, and mouth—are 50 or greater. In Figure 7, information processing system 1 considers the masking condition to be met and executes the masking process because the total numerical values ​​of the three parts of a person's face—eyes, nose, and mouth—are 50 or greater.

[0070] In Figure 6, the information processing system 1 performs a masking process on the person TG3 in the image (step S11). In Figure 6, the information processing system 1 performs a masking process in which it superimposes a black mask MK onto the area corresponding to the person's face, as shown in person TG3.

[0071] The above-described process is merely an example, and the information processing system 1 may perform various other processes. The information processing system 1 may recognize (identify) each person in the video when a person enters the field of view of the camera 21. For example, when the information processing system 1 detects a person's face in the video, an edge-side device such as the lighting device 100 may have a function to convert parts such as eyes, nose, and mouth into numerical values ​​and determine whether they are above a threshold. For example, when the lighting device 100 detects a person's face in the video, it converts parts such as eyes, nose, and mouth into numerical values.

[0072] For example, the information processing system 1 detects the movement of a person's head and neck using their skeleton, and when the camera determines that the face is visible, it performs masking on the face. As mentioned above, the entity performing the masking is not limited to the lighting device 100; other devices included in the information processing system 1, such as the server device 300 or the repeater 200, may also perform the masking. For example, in the information processing system 1, the lighting device 100 may transmit the captured video data to the server device 300, and the server device 300 may perform masking on the area where a person's face is visible.

[0073] Through the processing described above, the information processing system 1 can reduce the possibility of identifying individuals contained in the image. Therefore, the information processing system 1 can perform processing that takes into account the privacy of individuals. Note that the above is merely an example, and the information processing system 1 may perform various masking processes related to privacy protection. For example, in the information processing system 1, the camera 21 is installed on the ceiling, and the information processing system 1 detects faces using skeletal detection technology. For example, the information processing system 1 determines whether a person is looking upwards. The information processing system 1 quantifies each part of a person's face, such as the eyes, nose, and mouth, and if the numerical value meets predetermined conditions, it masks the image data.

[0074] Furthermore, if a person is simply looking upwards, the pixels may be coarse. The information processing system 1 quantifies the facial features and masks them if they meet predetermined conditions. The information processing system 1 does not need to perform any skeletal detection processing if the skeleton cannot be determined depending on the range of pixels. For example, if skeletal detection is impossible without parts of a person's body, the information processing system 1 does not need to perform any skeletal detection processing. The information processing system 1 may use any method, not limited to skeletal detection, as long as it can determine whether a person's face is likely to be captured.

[0075] Information processing system 1 may perform a masking process to hide a person's face in advance if it can estimate beforehand that the orientation (angle) of the person's face is likely to satisfy certain conditions. For example, information processing system 1 may derive an angle using the front as a reference, the ceiling, and the person's height, and then determine whether the person's face is likely to be visible based on the derived angle and the person's position. For example, information processing system 1 may identify the direction of the person's movement and estimate the orientation (direction) of the person's face. For example, if a person is moving in a direction where their face will not be visible, information processing system 1 does not need to perform a masking process on that person.

[0076] [Examples of processing related to privacy protection (data conversion examples)] Information processing system 1 performs data conversion as part of privacy protection processing. For example, information processing system 1 is a camera-equipped lighting system, and performs personal information protection processing on video data within the relay device 200 that sends it to the server device 300. This point will be explained below, including the prerequisites. Note that explanations of points similar to those mentioned above will be omitted as appropriate.

[0077] Conventionally, video footage captured by a camera is stored on a data server via a communication cable. However, if the captured video contains information that can identify an individual, such as their face or clothing, it may violate the Personal Information Protection Act, and there is room for improvement from the perspective of protecting personal information. Therefore, when a face is captured in an image (video) by the camera 21, the information processing system 1 digitizes parts such as the eyes, nose, and mouth, and converts this digitized data into attribute data indicating gender, age, etc., within the relay device 200 that sends it to the data server such as the server device 300, thereby reducing the possibility of an individual being identified. As a result, the information processing system 1 can reduce the risk of violating the Personal Information Protection Act.

[0078] For example, in Information Processing System 1, a lighting device 100 is installed on the ceiling of a factory or warehouse, and the captured data is sent to a data server such as a server device 300 via a relay device 200. When a person (for example, person TG in Figure 1) enters the field of view of the camera 21 (for example, area AR in Figure 1), Information Processing System 1 performs a skeleton detection process to detect the person's skeleton and recognizes the person based on the skeleton detection process. For example, if a person's face is captured in Information Processing System 1, the relay device 200 has a control unit 230 (processing unit 232) that digitizes facial features such as eyes, nose, and mouth from the video data and converts them into attribute data such as gender and age, and then processes and transmits it to the server device 300. In addition, Information Processing System 1 processes not only faces, but also items worn by the person, hairstyles, and their colors in a similar manner, as long as they can identify an individual.

[0079] Here, an example of data conversion processing will be explained using Figure 8. Figure 8 is a diagram illustrating an example of data conversion. For example, the repeater 200 performs the data conversion shown in Figure 8. Figure 8 shows the processing within the repeater 200. Note that the processing entity that performs data conversion is not limited to the repeater 200; other devices included in the information processing system 1, such as the server device 300 or the lighting device 100, may also perform data conversion processing.

[0080] The first information DT11 shown in Figure 8 contains personal information of a person. For example, the first information DT11 contains personal information of a person contained in an image captured by the camera 21. The first information DT11 includes information such as time, person ID, and numerical data of the eyes, nose, and mouth. The numerical data of the eyes, nose, and mouth shown in Figure 8 is generated by a numerical conversion process as shown in Figure 7.

[0081] Information processing system 1 performs a conversion process to convert first information, which represents a person's personal information, into second information, which represents a person's attribute information (step S21). In Figure 8, the repeater 200 performs a conversion process to convert first information DT11, which represents a person's personal information, into second information DT12, which represents a person's attribute information. For example, the repeater 200 converts first information DT11 to second information DT12 based on the person's ID, numerical data of eyes, nose, mouth, etc.

[0082] For example, if a person's ID is associated with attribute information indicating their gender, age, etc., the repeater 200 converts the first information DT11 to the second information DT12 based on that attribute information. For example, the repeater 200 estimates the age, gender, etc. corresponding to numerical data of the eyes, nose, and mouth, and converts the first information DT11 to the second information DT12 based on the estimated information. For example, the repeater 200 converts the first information DT11 to the second information DT12 using conversion information in which numerical data of the eyes, nose, and mouth are associated with attributes such as age and gender. For example, the repeater 200 converts the first information DT11 to the second information DT12 using conversion information in which the range of numerical data of the eyes, nose, and mouth is associated with each combination of attributes such as age (e.g., 30s) and gender. For example, the repeater 200 converts the numerical data in the first information DT11 to the age, gender, etc. corresponding to the attribute combination to which the numerical data belongs, thereby converting the first information DT11 to the second information DT12.

[0083] The second type of information DT12 shown in Figure 8 is attribute data that indicates a person's attributes. For example, the second type of information DT12 indicates a person's gender and age, but it is information that does not identify the individual corresponding to that data. The second type of information DT12 includes information on time, age, and gender. For example, the second type of information DT12 shown in Figure 8 indicates that the person in the image corresponding to that time is in their 30s and male.

[0084] In this way, the information processing system 1 performs data conversion as a process related to privacy protection. The information processing system 1 has an edge (lighting device 100) which integrates a camera and a lighting fixture, and a relay device 200 which sends the data acquired by the lighting device 100 to the server device 300. If a face is visible in the captured video, the information processing system 1 converts the facial features into numerical data. In the example above, the information processing system 1 uses the relay device 200 to convert the numerical data of the facial features into attribute data such as age and gender, and then transmits the data with protected personal information to the server device 300.

[0085] Next, we will explain the information processing flow related to data conversion in the information processing system 1. Figure 9 is a flowchart showing an example of data conversion processing by the information processing system. In the following explanation, the information processing system 1 will be described as the processing entity, but the processing shown in Figure 9 may be performed by any of the devices, such as the lighting device 100, the repeater 200, or the server device 300, depending on the device configuration included in the information processing system 1.

[0086] Information processing system 1 recognizes when a person enters the field of view (step S101). For example, lighting device 100 recognizes when a person enters the field of view of camera 21.

[0087] The information processing system 1 determines whether or not a face is visible (step S102). For example, the repeater 200 determines whether or not a person's face is visible in the image captured by the camera 21.

[0088] If the information processing system 1 determines that a face is visible (step S102: Yes), it converts the facial features into numerical values ​​(step S103). For example, if the repeater 200 determines that a face is visible in the image captured by the camera 21, it converts the facial features into numerical values.

[0089] Then, the information processing system 1 converts the data into attributes (step S104). For example, the repeater 200 converts the numerical values ​​obtained by converting the facial features in the image captured by the camera 21 into attributes. Then, the information processing system 1 saves the data to the cloud (step S105). For example, the repeater 200 sends the attribute-converted information to the server device 300, and the server device 300 saves the received attribute-converted information.

[0090] If the information processing system 1 determines that no face is visible (step S102: No), it proceeds to step S105 without performing steps S103 and S104. For example, the repeater 200 transmits the image captured by the camera 21 to the server device 300, and the server device 300 saves the received image.

[0091] Through the processing described above, the information processing system 1 can reduce the possibility of identifying individuals contained in images. Therefore, the information processing system 1 can perform processing that takes into account the privacy of individuals. Note that the above is merely an example, and the information processing system 1 may perform various privacy-protection conversion processes. For example, if the information processing system 1 identifies personal information on the gateway side, such as the relay device 200, it may convert it into attributes and output it, and store it in the cloud, such as the server device 300. For example, the information processing system 1 may analyze the image and output the numerical values ​​of people's movements. For example, if the information processing system 1 recognizes a face, it may convert it into attribute information such as gender and age. For example, if the information processing system 1 detects information that is presumed to be able to identify an individual, it may convert it into attributes.

[0092] For example, Information Processing System 1 detects a face from an image and quantifies each facial feature. If the numerical values ​​of the facial features satisfy predetermined conditions, Information Processing System 1 estimates attribute values ​​indicating the person's attributes, converts them, and outputs them. For example, if the numerical values ​​of a person's facial features are sufficient to identify that person, Information Processing System 1 estimates attribute values, converts the numerical values ​​of the facial features into attribute values, and outputs them. Note that a person's attributes are not limited to gender, age, etc., but may also include things like wearing glasses, wearing certain equipment such as a suit, owning possessions, or not owning possessions.

[0093] The information processing system 1 may generate additional information including at least one of interest information indicating a person's interests and movement information indicating the direction of a person's movement. For example, the repeater 200 may generate additional information including at least one of interest information indicating a person's interests and movement information indicating the direction of a person's movement. For example, the information processing system 1 may detect a person's orientation (gaze). In this case, the information processing system 1 may estimate that an object in the direction of the estimated person's orientation (gaze) is an object that the person is interested in.

[0094] For example, the information processing system 1 may detect a person's orientation and estimate their direction of movement. In this case, the information processing system 1 may estimate that an object located in the direction of the estimated movement of the person is an object of interest to that person. For example, the information processing system 1 may detect and add information that describes the user's situation.

[0095] For example, information processing system 1 may encrypt and communicate information. For example, in information processing system 1, the repeater 200 may encrypt and communicate information with the lighting device 100, the server device 300, etc. Any encryption method can be used, and information processing system 1 may use various encryption methods to communicate encrypted information.

[0096] [Examples of privacy protection processing (filtering processing examples)] Information processing system 1 performs filtering as part of privacy protection processing. For example, information processing system 1 performs processing related to personal information protection by analyzing and filtering video data from camera 21. This point will be explained below, including the prerequisites. Note that explanations of points similar to those mentioned above will be omitted as appropriate.

[0097] Traditionally, filtering processes for individuals in video data have involved filtering out faces or entire bodies to protect personal information. However, when individuals possess personal belongings such as pets, they may be identified from that information. Therefore, Information Processing System 1 automatically changes the filtering range and strength according to the risk of personal identification, thereby reducing the possibility of identifying individuals from information outside the filtering range. This reduces the risk of Information Processing System 1 violating the Personal Information Protection Act.

[0098] First, Figure 10 will be used to explain the components related to filtering in the information processing system 1. Figure 10 is a diagram showing an example of the components related to filtering. The information processing system 1 performs filtering processing using components such as camera DV1 and specific device DV2.

[0099] Camera DV1 is a component that captures images and corresponds to, for example, a lighting device 100 having a camera 21. Specific device DV2 is a component that has an analysis unit, a filtering unit, a storage unit, etc., and holds attribute data and processed video data, and corresponds to, for example, a server device 300. The analysis unit and filtering unit in Figure 10 correspond to the processing unit 332 of the server device 300. The storage unit in Figure 10 corresponds to the storage unit 320 of the server device 300. The external output in Figure 10 is a configuration that outputs from specific device DV2 to an external device and corresponds to, for example, the communication unit 310 of the server device 300.

[0100] For example, when the information processing system 1 detects a person with camera DV1, it analyzes the person using an analysis device such as server device 300 (identification device DV2 in Figure 10) and saves the results as attribute data to a storage device. Note that the analysis device (identification device DV2) is not limited to server device 300; it may also be a lighting device 100 or a repeater 200, etc., as long as it is a device that performs filtering. The information processing system 1 then analyzes the area around the person using the analysis device and determines the content of the filtering process based on the analysis results and a predefined risk of personal identification. The information processing system 1 also saves the filtered video information to a storage device such as the storage unit 320 of server device 300 (storage unit of identification device DV2 in Figure 10).

[0101] Here, we will explain an example of attribute data held by the specific device DV2. Figure 11 is a diagram showing an example of attribute data. As shown in Figure 11, the attribute data includes time and coordinates in the video and is linked to a specific individual in the processed video information. For example, the attribute data shown in Figure 11 includes information on time, coordinates, gender, and age. For example, the attribute data shown in Figure 11 shows the attributes such as gender and age of a person included in the area indicated by the coordinates in the image corresponding to that time.

[0102] Information processing system 1 may perform the following filtering processes related to privacy protection. Information processing system 1 may define personal identification risks, increase the risk level each time a risk is violated, and perform predefined filtering processes according to the level. For example, information processing system 1 may define a default filter at risk level 0 and change the filter as the level increases. For example, information processing system 1 may define personal identification risks and perform predefined filtering processes according to the violated personal identification risks. The processing according to the risk level will be described in detail in Figure 12.

[0103] For example, information processing system 1 may define personal identification risks and dynamically change the definition of filtering processing based on the personal identification risks that are encountered. For example, specific examples of personal identification risks and processing include the following: For example, information processing system 1 may perform filtering processing that increases the filtering strength, such as blurring, for physical characteristics of an individual, such as hairstyle and height. In addition, information processing system 1 may perform filtering processing that corrects the color of the filtered area, such as converting to grayscale, for color information such as clothing and skin color.

[0104] Furthermore, for example, the information processing system 1 may perform filtering processing that intentionally reduces the frame rate of the filter unit for motion characteristics such as walking style. For example, the information processing system 1 may perform filtering processing that expands the filtering range by referring to the location of specific risks for personal belongings such as large luggage and wheelchairs. For example, the information processing system 1 may perform filtering processing such as expanding the filtering range and reducing the frame rate for things other than people that can be linked to an individual, such as pets. The processing according to the risk content will be described in detail in Figure 13.

[0105] The above is merely one example, and the information processing system 1 may perform various privacy-protection filtering processes. For example, the information processing system 1 may estimate how easily an individual can be identified based on their possessions, physical characteristics, movement characteristics, color, etc., and perform filtering in accordance with the estimation results. For example, the information processing system 1 may perform processes such as removing color, changing the size of the mosaic, blacking out, dropping frames, dropping frames around the edges, reducing gradation, or hiding possessions, depending on the estimation results of how easily an individual can be identified.

[0106] For example, information processing system 1 may perform filtering by blacking out parts of an image if the estimated value of the ease of identifying an individual is above a predetermined threshold. Alternatively, information processing system 1 may perform filtering using a whitelist of items that will not be filtered or a blacklist of items that will be filtered. If the part of an image held by a person is included in the blacklist of items that will be filtered, information processing system 1 may perform filtering by blacking out that item.

[0107] Information processing system 1 performs dynamic filtering of the area around a person. In the example described above, camera DV1 is placed in a predetermined position and captures images of a specific space. The analysis unit performs person detection and analysis of the surrounding area based on the video information captured by camera DV1, and outputs the analysis information to the storage unit and the video information to the filtering unit. The filtering unit processes the video information based on the analysis results and outputs the processed video information to the storage unit. The storage unit stores the data output from the analysis unit and the filtering unit.

[0108] More specifically, camera DV1 outputs spatial video information to the analysis unit. The analysis unit then determines the risk level based on the risk of personal identification from the spatial video information and outputs the video information and the determined risk level information to the filtering unit. The filtering unit performs filtering processing on the video information according to the risk level. Based on the video information, the analysis unit determines whether each individual identification risk is violated, determines the filtering range and intensity according to each violated individual identification risk, and outputs the video information and the filtering unit. The filtering unit performs the filtering processing determined by the analysis unit. Through the above processing, the information processing system 1 can reduce the possibility of identifying individuals contained in the image. Therefore, the information processing system 1 can perform processing that takes into account the privacy of individuals.

[0109] Next, we will explain the information processing flow related to filtering in Information Processing System 1. Figure 12 is a flowchart showing an example of filtering based on risk level. In the following explanation, Information Processing System 1 will be described as the processing entity, but the processing shown in Figure 12 may be performed by any of the devices, such as the lighting device 100, repeater 200, or server device 300, depending on the device configuration included in Information Processing System 1.

[0110] The information processing system 1 determines whether or not there is a person in the video (step S201). For example, the server device 300 determines whether or not there is a person in the image (video) captured by the camera 21. If the information processing system 1 determines that there is no person in the video (step S201: No), it repeats the process of step S201.

[0111] If the information processing system 1 determines that there is a person in the video (step S201: Yes), it defines the person in the video as [A], defines a [risk level] of 0 for [A], and defines [attribute data A] (step S202). For example, if the server device 300 determines that there is a person in the image (video) captured by the camera 21, it defines that person as user UA, initializes the risk level LA of user UA to 0, and defines the attribute data of user UA as attribute data DA.

[0112] Information processing system 1 performs a person analysis (step S203). For example, server device 300 analyzes the person contained in the image by analyzing the image captured by camera 21.

[0113] The information processing system 1 stores the analysis results in [Attribute Data A] of [A] (step S204). For example, the server device 300 stores the analysis results showing attributes such as the age and gender of the user UA in the user UA's attribute data DA.

[0114] Information processing system 1 performs analysis of the area surrounding the person (step S205). For example, server device 300 performs analysis of the area surrounding the person included in the image by analyzing the image captured by camera 21.

[0115] The information processing system 1 determines whether or not it violates personal identification risk 1 (step S206). For example, the server device 300 determines whether or not the analysis result violates the first personal identification risk in the list of personal identification risks stored in the storage unit 320.

[0116] If the information processing system 1 finds that the result of user UA's analysis matches the first personal identification risk in the list of personal identification risks stored in the memory unit 320, it adds 1 to the risk level LA of user UA.

[0117] If the information processing system 1 does not violate personal identification risk 1 (step S206: No), it proceeds to step S208 without performing the processing in step S207.

[0118] The information processing system 1 determines whether or not it violates personal identification risk N (step S208). For example, the server device 300 determines whether or not the analysis result violates the Nth personal identification risk in the list of personal identification risks stored in the storage unit 320.

[0119] If the information processing system 1 finds that the result of user UA's analysis conflicts with the Nth personal identification risk in the list of personal identification risks stored in the memory unit 320, it increments the risk level LA of user UA by 1.

[0120] If the information processing system 1 does not violate the personal identification risk N (step S208: No), it proceeds to step S210 without performing the processing in step S209. Note that Figure 12 only illustrates the processing flow for personal identification risk 1 and personal identification risk N, but if N is 3 or greater, the risk levels for the 2nd to N-1th personal identification risks are also counted in the same way as in steps S206, S207, etc.

[0121] Information processing system 1 determines whether the [risk level] is 0 or not (step S210). For example, server device 300 determines whether the risk level LA of user UA is 0 or not.

[0122] If the [risk level] is 0 (step S210: Yes), the information processing system 1 performs default filtering (step S211). For example, if the risk level LA of user UA is 0, the server device 300 performs default filtering on the images taken by user UA. For example, if the risk level LA of user UA is 0, the server device 300 performs filtering to change the size of the mosaic.

[0123] If the [risk level] is not 0 (step S210: No), the information processing system 1 determines whether the [risk level] is N or not (step S212). For example, if the risk level LA of user UA is not 0, the server device 300 determines whether the risk level LA of user UA is N or not.

[0124] If the [risk level] is N (step S212: Yes), the information processing system 1 performs level N filtering (step S213). For example, if the risk level LA of user UA is N, the server device 300 performs level N filtering on images taken of user UA. For example, if the risk level LA of user UA is N, the server device 300 performs filtering by blacking out the area in the image that contains user UA.

[0125] If the [risk level] is not N (step S212: No), the information processing system 1 terminates processing without performing step S213.

[0126] Next, we will explain the information processing flow related to filtering in Information Processing System 1. Figure 13 is a flowchart showing an example of filtering based on risk content. In the following explanation, Information Processing System 1 will be described as the processing entity, but the processing shown in Figure 13 may be performed by any of the devices, such as the lighting device 100, repeater 200, or server device 300, depending on the device configuration included in Information Processing System 1.

[0127] The information processing system 1 determines whether or not there is a person in the video (step S301). For example, the server device 300 determines whether or not there is a person in the image (video) captured by the camera 21. If the information processing system 1 determines that there is no person in the video (step S301: No), it repeats the process of step S301.

[0128] If the information processing system 1 determines that there is a person in the video (step S301: Yes), it defines the person in the video as [A], sets the [risk level] of [A] to 0, and defines [attribute data A] (step S302). For example, if the server device 300 determines that there is a person in the image (video) captured by the camera 21, it defines that person as user UA, initializes the risk level LA of user UA to 0, and defines the attribute data of user UA as attribute data DA.

[0129] Information processing system 1 performs a person analysis (step S303). For example, server device 300 analyzes the person contained in the image by analyzing the image captured by camera 21.

[0130] The information processing system 1 stores the analysis results in [Attribute Data A] of [A] (step S304). For example, the server device 300 stores the analysis results showing attributes such as the age and gender of the user UA in the attribute data DA of the user UA.

[0131] Information processing system 1 performs analysis of the area surrounding the person (step S305). For example, server device 300 performs analysis of the area surrounding the person included in the image by analyzing the image captured by camera 21.

[0132] The information processing system 1 determines whether or not there is a moving object following the person (step S306). For example, the server device 300 determines whether or not there is a moving object following the person in the image based on the analysis results of the area around the person in the image.

[0133] If there is a moving object following a person (step S306: Yes), the information processing system 1 expands the filter range (step S307). For example, if there is an object following the user UA in the image, the server device 300 expands the filter range from the range of the user UA to the range that includes the object.

[0134] If there is no moving object following the person (step S306: No), the information processing system 1 proceeds to step S308 without performing the processing in step S307.

[0135] The information processing system 1 determines whether the color information is a color other than the defined color (step S308). For example, the server device 300 compares the color indicated by the color definition information related to filtering stored in the storage unit 320 with the color indicated by the color information included in the analysis result, and determines whether the color indicated by the color information is a color other than the color indicated by the definition information.

[0136] If the color information is not a defined color (step S308: Yes), the information processing system 1 corrects it to the color information of the filter unit (step S309). For example, if the color indicated by the color information is not a color indicated by the color definition information for filtering stored in the storage unit 320, the server device 300 corrects the area in the image corresponding to the color indicated by the color information to the color indicated by the color definition information for filtering.

[0137] If the color information is not a defined color (step S308: No), the information processing system 1 proceeds to step S310 without performing the processing in step S309.

[0138] The information processing system 1 determines whether or not there are other filter definitions (step S310). For example, the server device 300 determines whether or not there are any filtering definitions other than color stored in the storage unit 320.

[0139] If there are other filter definitions (step S310: Yes), the information processing system 1 executes the corresponding processing (step S311). For example, if the server device 300 has a filter definition to hide a person's possessions, it performs filtering to black out objects owned by the user UA.

[0140] If there are no other filter definitions (step S310: No), the information processing system 1 terminates processing without performing step S311.

[0141] [Examples of privacy protection procedures (Examples of privacy protection specifications)] Information processing system 1 performs privacy protection-related processing, including specifying privacy protection. For example, information processing system 1 performs video data processing for a camera-equipped lighting system. This point will be explained below, including the prerequisites.

[0142] Traditionally, lighting fixtures with cameras have been installed on the ceiling and film directly downwards, so the proportion of scenes that can identify individuals is not high. However, depending on a person's posture or the direction of their face, there are scenes in which individuals can be identified. For example, depending on the type of installation (business), these images may require special handling from the perspective of personal information protection laws.

[0143] For example, one application of video from a camera-equipped lighting system is in the sports field. A gymnasium, for instance, would be a suitable location for analyzing individual player movements and formations using cameras installed there. Teams and players motivated by the desire to analyze and improve their team performance would understand and accept being filmed. On the other hand, some people, such as spectators, would not want to be filmed.

[0144] Therefore, the information processing system 1 can protect privacy by applying a masking process to the video. In the information processing system 1, which also functions as a camera-equipped lighting system, the captured video is stored on a cloud server (for example, server device 300). The server device 300 saves and makes the video downloadable, and also performs analysis on the acquired video by incorporating an image analysis engine.

[0145] For example, Information Processing System 1 recognizes people by detecting their skeletons. Because Information Processing System 1 captures images directly below the ceiling, the percentage of images that can identify individuals is not high, but depending on a person's posture and the direction of their face, it may be possible to identify them. For example, in the case of a gymnasium of a team that wants to improve its sports performance, these images are often understood as part of an effort to improve the team, but this may not be understood by spectators watching the sports. In such cases, Information Processing System 1 may also need to take measures (privacy protection).

[0146] Therefore, the information processing system 1 receives a specification regarding the protection of personal information and performs processing on the image in accordance with the received specification regarding the protection of personal information. In the following description, the information processing system 1 is described as the processing entity, but the processing shown below may be performed by any of the devices, such as the lighting device 100, the repeater 200, or the server device 300, depending on the device configuration included in the information processing system 1.

[0147] Information processing system 1 accepts a specification that includes a mode of whether or not to perform processing related to the protection of personal information. For example, information processing system 1 accepts a specification that includes the area to be subject to the protection of personal information. For example, information processing system 1 accepts a specification that includes the person to be subject to the protection of personal information. For example, information processing system 1 accepts a specification of a person's state that includes at least one of a person's attributes, movements, and appearance.

[0148] Information processing system 1 performs processing on images captured by a camera in accordance with the specifications regarding the protection of personal information received for those images. For example, information processing system 1 performs processing regarding the protection of personal information on images in accordance with the specifications, which include a mode for whether or not to perform processing regarding the protection of personal information. If the mode is a protection mode for performing processing regarding the protection of personal information, then information processing system 1 performs processing regarding the protection of personal information on the images.

[0149] Information processing system 1 performs processing related to the protection of personal information, such as masking and filtering, on areas of an image that include areas to be protected. For example, information processing system 1 performs processing to black out areas of an image that include areas to be protected.

[0150] Information processing system 1 performs personal information protection processes, such as masking and filtering, on images that include individuals whose personal information is to be protected, according to the specified information. For example, information processing system 1 performs a process to black out individuals in images according to the specified information that includes individuals whose personal information is to be protected.

[0151] Information processing system 1 performs personal information protection processes, such as masking and filtering, on people in an image who correspond to a specified state of a person, which includes at least one of a person's attributes, movements, and appearance. For example, information processing system 1 performs a process to black out people in an image who correspond to a specified state of a person, which includes at least one of a person's attributes, movements, and appearance.

[0152] Thus, the information processing system 1 is a system that accepts a setting for whether or not to perform processing related to the protection of personal information on the video images captured by the camera 21. As described above, the processing related to the protection of personal information may be performed by any of the devices of the information processing system 1, such as the lighting device 100, the repeater 200, or the server device 300.

[0153] For example, information processing system 1 accepts a specification of whether or not to perform processing related to the protection of personal information. For example, information processing system 1 accepts a specification of the area to be subject to the protection of personal information. Information processing system 1 accepts a specification of a rectangular area to be subject to the protection of personal information. For example, information processing system 1 accepts a specification of a subject area such as a user in a certain area and performs masking processing on the subject area. For example, information processing system 1 may identify a sports area (target area) by recognizing white lines and perform processing related to the protection of personal information on that target area.

[0154] For example, information processing system 1 may estimate the attributes of a person (subject) included in an image, such as whether they are a player or a spectator, and perform processing related to the protection of personal information for spectators. For example, information processing system 1 may estimate whether a person (subject) included in an image is wearing a mask, estimate that a person (subject) wearing a mask is sitting, and perform processing related to the protection of personal information for that person. For example, information processing system 1 may estimate the appearance of a person (subject) included in an image, such as the color of their uniform, and if their appearance falls under the category of personal information protection, perform processing related to the protection of personal information for that person.

[0155] For example, when the information processing system 1 performs masking on an image, such as blacking out or blurring a person, it may perform the masking on the area corresponding to the person's face. For example, when the information processing system 1 performs masking only on the area corresponding to the person's face, if the person is recognizable, it may also perform the masking on an area that includes areas other than the person's face, such as the entire outline of the person. For example, the information processing system 1 may detect the skeleton and convert the outline of a person in the image into an extremely stylized shape (e.g., a stick figure). In this case, if the information processing system 1 estimates that the person is male, it may convert that person into a blue stick figure, and if the person estimates that the person is female, it may convert that person into a red stick figure.

[0156] Through the processing described above, the information processing system 1 can reduce the possibility of identifying individuals contained in the image. Therefore, the information processing system 1 can perform processing that takes into account the privacy of individuals. In addition, the information processing system 1 may provide a system that takes usability into consideration, such as reducing the effort of setting up by utilizing video analysis, rather than simply specifying an area and performing masking. For example, instead of performing masking by specifying an area in a unambiguous way, the information processing system 1 may prepare various modes based on image analysis. For example, even when specifying an area, the information processing system 1 may not only allow a person to manually specify an area in the video, but may also, for example, recognize the white lines of a court and assume that people within the white lines are players, and therefore not perform masking on people in that area, while processing other people. Also, since spectators are often seated, the information processing system 1 may perform privacy protection processing such as masking on seated people.

[0157] For example, the information processing system 1 may focus on the color of the uniform as a characteristic of the players and use that as a trigger to perform processing related to privacy protection. For example, in masking, the information processing system 1 may not only represent the information as black, but also display it as, for example, an anthropomorphic icon. In that case, for example, the information processing system 1 may perform audience mobilization analysis by color-coding the information, for example, blue for men and red for women. The above processing in the information processing system 1 may be performed in the cloud, such as on a server device 300, or on the edge (camera) side, such as on a lighting device 100.

[0158] [Other system configuration examples] The configuration of the information processing system 1 described above is merely an example, and the information processing system 1 can employ any device configuration as long as the desired processing is possible. For example, the information processing system 1 may have a configuration without a repeater 200. In this case, the privacy protection processing that the repeater 200 would perform may be performed by the server device 300 or the lighting device 100.

[0159] While embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. Furthermore, these embodiments and their variations can be combined as appropriate, as long as the processing content is not contradictory. [Explanation of Symbols]

[0160] 1. Information Processing System 100 Lighting devices 10 Communications Department 20 Camera Units 30 lighting units 40 Control Unit 41 Acquisition Department 42 Processing Units 43 Transmitter 50 Storage section 200 Repeaters 210 Communications Department 220 Storage section 230 Control Unit 231 Acquisition Department 232 Processing Unit 233 Transmitter 300 Server Devices 310 Communications Department 320 Storage section 330 Control Unit 331 Acquisition Department 332 Processing Unit 333 Transmitter 400 display device

Claims

1. A processing unit that performs a conversion process to convert first information indicating a person's personal information contained in an image captured by a camera into second information indicating the person's attribute information, performing default filtering on the image if the risk level regarding the risk of personal identification is 0, performing filtering to black out the person in the image if the risk level is greater than 0 and less than the predetermined value, and performing no filtering if the risk level is greater than 0 and less than the predetermined value. It has, The aforementioned processing unit, An information processing system that generates additional information including interest information indicating an object located ahead of the direction of movement of the person and of which the person is interested.

2. The aforementioned processing unit, The conversion process is performed to convert the first information, which includes at least one of facial information showing the person's face and identification information identifying the person, into second information, which includes at least one of the person's age and gender. The information processing system according to claim 1.

3. The aforementioned processing unit, The conversion process is performed to convert the first information, which includes numerical values ​​converted for each part of the person's face included in the image, including at least one of the eyes, nose, and mouth, into second information. The information processing system according to claim 2.

4. It comprises an edge equipped with a light source and the camera, and an information processing device equipped with the processing unit, The aforementioned information processing device is The conversion process is executed to convert the first information of the person contained in the image acquired from the edge into the second information of the person. The information processing system according to any one of claims 1 to 3.

Images