Information processing system, storage medium, and information processing method

Abstract

The invention provides an information processing system, a storage medium and an information processing method. An information processing system is provided with a processor that performs: a process for acquiring image data indicating a captured image; and generating additional information of the image data without including identification information related to an unallowed object when the unallowed object, which is not allowed to provide identification information for identifying an individual, exists in the image.

Description

Technical Field

[0001] The invention relates to an information processing system, a storage medium and an information processing method. Background Art

[0002] Patent document 1 proposes an image monitoring system, which includes: a camera mechanism that sends camera conditions from a monitored location and performs camera recording; a recognition mechanism that recognizes a human body based on camera information output by the camera mechanism; an abstraction mechanism that abstracts information related to the human body based on the output of the recognition mechanism; a synthesis mechanism that synthesizes the output of the abstraction mechanism and the output of the camera mechanism; and an information transmission line that transmits the output of the synthesis mechanism to the monitored location.

[0003] Patent document 2 proposes a location information system, which includes a server having a function of grasping the position of a mobile terminal or portable transmitter that performs short-range communication with a transmitter or / and a receiver based on received inherent information of the transmitter or receiver, the server including: an area setting unit that sets a privacy protection area in any area of ​​a specified area within a management area; an area determination unit that grasps the position of a mobile terminal or portable transmitter that performs short-range communication with a transmitter or / and a receiver based on the inherent information of the transmitter or the receiver, and determines whether the grasped position is within the privacy protection area; and an information sending unit that sends the grasped position of the mobile terminal or portable transmitter and map information of the management area including the position and determination result information of the area determination unit, and when it is normally determined that the position of the mobile terminal or portable transmitter is within the privacy protection area, it is set that the personal information of the holder of the mobile terminal or portable transmitter cannot be read, and on the other hand, when it is determined that the position of the mobile terminal or portable transmitter is within the privacy protection area in an emergency, it is set that the personal information of the holder of the mobile terminal or portable transmitter can be read.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2002-142214

[0005] Patent Document 2: Japanese Patent Application Publication No. 2023-006192

[0006] Among systems for acquiring information from images of space, there is known an information system that allows viewing of personal information in an emergency. In such an information system, since the personal information of all persons can be viewed, there is a problem that even the personal information of persons who have not given permission can be viewed. Such a problem is not limited to information related to persons, but also applies to information related to animals (e.g., pets) or property (e.g., robots). Summary of the invention

[0007] Therefore, an object of the present invention is to provide an information processing system, a storage medium, and an information processing method that can prevent the generation of information that can identify an individual for an unauthorized object.

[0008] In order to achieve the above-mentioned purpose, the information processing system involved in the first method has a processor, which performs the following processing: acquiring image data representing the captured image; and when an unauthorized object that is not allowed to provide identification information of an identifiable individual is reflected in the image, generating additional information of the image data without including the identification information related to the unauthorized object.

[0009] According to the information processing system involved in the first embodiment, in the information processing system involved in the second embodiment, when a permitted object that is allowed to provide the identification information is reflected in the image, the processor generates the additional information including the identification information related to the permitted object.

[0010] According to the information processing system involved in the second embodiment, in the information processing system involved in the third embodiment, when a predetermined exception occurs and the permitted object is reflected in the image, the processor generates the additional information including the identification information related to the permitted object.

[0011] In the information processing system according to the third aspect, in the information processing system according to the fourth aspect, the processor detects the occurrence of the abnormality using the image data.

[0012] In the information processing system according to the third aspect or the fourth aspect, in the information processing system according to the fifth aspect, the processor performs the following processing: receiving sensor data; and detecting the occurrence of the abnormality using the sensor data.

[0013] According to the information processing system involved in any one of the third to fifth methods, in the information processing system involved in the sixth method, the processor generates the additional information without including the identification information related to the unauthorized object and the permitted object when the abnormality does not occur.

[0014] According to the information processing system according to any one of the first to sixth aspects, in the information processing system according to a seventh aspect, the processor transmits the supplementary information to the outside via a communication means.

[0015] In the information processing system according to the seventh aspect, in the information processing system according to the eighth aspect, the processor displays the supplementary information transmitted to the outside on a preset display means.

[0016] According to the information processing system involved in any one of the first to eighth aspects, in the information processing system involved in the ninth aspect, the additional information includes at least any one of skeleton information related to the object or object information related to an object other than the object.

[0017] In the information processing system according to the ninth aspect, in the information processing system according to a tenth aspect, the skeleton information or the object information includes recognition accuracy information recognized from the image data.

[0018] In the information processing system according to any one of the first to tenth aspects, in the information processing system according to the eleventh aspect, the identification information includes at least one of a name of the object or an image showing the object.

[0019] The storage medium involved in the 12th method stores an information processing program that enables a computer to perform the following processing: acquiring image data representing a captured image; and when an unauthorized object that is not allowed to provide identification information of an identifiable individual is reflected in the image, generating additional information of the image data without including the identification information related to the unauthorized object.

[0020] The information processing method involved in the 13th mode includes the following steps: acquiring image data representing the captured image; and when an unauthorized object that is not allowed to provide identification information of an identifiable individual is reflected in the image, generating additional information of the image data without including the identification information related to the unauthorized object.

[0021] Effects of the Invention

[0022] According to the information processing system according to the first aspect, it is possible to provide an information processing system capable of preventing the generation of information that can identify an individual for an unauthorized object.

[0023] According to the information processing system according to the second aspect, it is possible to prevent information that can identify an individual from being generated for an unauthorized object and to generate information that can identify an individual for an authorized object.

[0024] According to the information processing system according to the third aspect, when information capable of identifying an individual is generated, occurrence of a preset abnormality can be used as a condition.

[0025] According to the information processing system according to the fourth aspect, when the occurrence of abnormality is detected, the required data can be reduced compared to the case where data other than image data is used.

[0026] According to the information processing system according to the fifth aspect, when detecting the occurrence of an abnormality, it is possible to detect the occurrence of various abnormalities compared to the case of using image data or the like.

[0027] According to the information processing system according to the sixth aspect, when no abnormality occurs, it is possible to prevent the generation of information that can identify an individual regardless of whether it is permitted or not.

[0028] According to the information processing system according to the seventh aspect, a service using the additional information can be provided to the destination of the additional information.

[0029] According to the information processing system according to the eighth aspect, it is possible to visualize what the transmitted incidental information is.

[0030] According to the information processing system according to the ninth aspect, information on the degree to which the individual cannot be identified with respect to the object or information on an object different from the object can be generated by including it in the supplementary information regardless of whether it is permitted or not.

[0031] According to the information processing system according to the tenth aspect, it is possible to notify the reliability level of information recognized from image data.

[0032] According to the information processing system according to the eleventh aspect, it is possible to prevent the generation of information that is highly likely to identify an individual for an unauthorized object.

[0033] According to the storage medium according to the twelfth aspect, it is possible to provide a storage medium capable of preventing the generation of information capable of identifying an individual for an unauthorized object.

[0034] According to the information processing method according to the thirteenth aspect, it is possible to provide an information processing method capable of preventing the generation of information that can identify an individual for an unauthorized object. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Embodiments of the present invention will be described in detail with reference to the following drawings.

[0036] Figure 1 is a diagram showing an example of a schematic configuration of an information providing system 10 according to the present embodiment;

[0037] Figure 2 is a diagram showing an example of a hardware configuration of an information processing system 100 according to the present embodiment;

[0038] Figure 3 is a diagram showing an example of a functional configuration of an information processing system 100 according to the present embodiment;

[0039] Figure 4is a diagram showing an example of registration information stored by the information processing system 100 according to the present embodiment;

[0040] Figure 5 is a diagram showing an example of the flow of information processing performed by the information processing system 100 according to the present embodiment;

[0041] Figure 6 An example of supplementary information generated by the information processing system 100 according to the present embodiment is shown. DETAILED DESCRIPTION

[0042] Hereinafter, an example of an embodiment of the present invention will be described with reference to the accompanying drawings. In addition, in each of the accompanying drawings, the same reference symbols are given to the same or equivalent components and parts. In addition, for the convenience of description, the dimensional ratios of the accompanying drawings are exaggerated and sometimes differ from the actual ratios.

[0043] Figure 1 1 is a diagram showing an example of a schematic configuration of an information providing system 10 according to the present embodiment. The information providing system 10 includes a communication mechanism 20 , a cloud system 40 , and an information processing system 100 .

[0044] The communication mechanism 20 can connect multiple computers in a communicative manner. In this figure, the communication mechanism 20 connects the cloud system 40 and the information processing system 100. For example, the communication mechanism 20 can be various networks such as LAN, WAN, Internet or intranet. In addition, the communication mechanism 20 can connect multiple computers wirelessly or by wire.

[0045] The cloud system 40 is a computer system that applies cloud computing. The cloud system 40 provides various services to users via the communication mechanism 20. In this embodiment, the cloud system 40 obtains additional information generated based on image data of a space 60 described later from the information processing system 100, and provides various services to users using the additional information.

[0046] Space 60 is a space that becomes a photographic object. In this figure, space 60 is a store, and as an example, a situation in which there are a plurality of objects 80A, 80B, and 80C (collectively referred to as "objects 80" when no distinction is required) that are people in the store is shown. In addition, space 60 is not limited to a store, but may be any space where images can be taken, such as an office, a factory, a hospital, or a medical site. Furthermore, object 80 is not limited to a person, but may be any object such as an animal (for example, a pet) or a property (for example, a robot) that may cause a problem in generating information that can identify an individual without permission.

[0047] The information processing system 100 acquires image data representing an image of the space 60, and generates incidental information of the image data. Here, when the incidental information is generated based on the image data of the space 60, identification information that can identify an individual, such as personal information or private information, is generated without permission, which may become a problem. Therefore, the information processing system 100 involved in this embodiment generates incidental information without including identification information related to the unauthorized object 80 that is not allowed to provide identification information. This will be described in detail.

[0048] Figure 2 1 is a diagram showing an example of the hardware configuration of the information processing system 100 according to the present embodiment. The information processing system 100 includes a CPU 121, a ROM 122, a RAM 123, a storage device 124, a communication interface 125, a user interface 126, a camera 127, and a sensor 128. These configurations are connected via a bus 129 so as to be able to communicate with each other.

[0049] The CPU 121 is a central processing unit that executes various programs and controls various structures. The ROM 122 stores various programs and various data. The RAM 123 temporarily stores programs or data as a work area. The storage device 124 is composed of a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

[0050] In the information processing system 100 according to the present embodiment, the information processing program is stored in the ROM 122 or the storage device 124. The CPU 121 reads the target program from the ROM 122 or the storage device 124 and executes it using the RAM 123 as a work area. The CPU 121 controls each component and performs various calculations according to the program.

[0051] The communication interface 125 is an interface for the information processing system 100 to communicate with other devices. The user interface 126 is an input and output interface for the information processing system 100 to exchange information with a user. For example, the user interface 126 may include a keyboard, a mouse, and a microphone as input devices, and a monitor and a speaker as output devices.

[0052] The camera 127 captures an image of the space 60 using an imaging device such as a CCD or a CMOS. The term "image" may be interpreted as including both a still image and a moving image. The camera 127 outputs image data representing an image of the space 60 captured.

[0053] The sensor 128 measures various physical quantities. For example, the sensor 128 may be a fire sensor, an earthquake sensor, or a pulse sensor. The sensor 128 outputs a measurement value obtained by measuring a physical quantity or a calculation result obtained by performing a predetermined calculation on the measurement value as sensor data.

[0054] In addition, in the above description, the case where the sensor 128 is provided inside the information processing system 100 is shown as an example, but the present invention is not limited thereto. The sensor 128 may be provided outside the information processing system 100. In this case, the sensor 128 may be connected to the information processing system 100 via the communication mechanism 20 so as to be communicable.

[0055] Figure 3 1 is a diagram showing an example of the functional configuration of the information processing system 100 according to the present embodiment. The information processing system 100 includes an acquisition unit 131, an analysis unit 132, a detection unit 133, an inquiry unit 134, a generation unit 135, and a transmission unit 136. These functional configurations are realized by the CPU 121 reading the information processing program from the ROM 122 or the storage device 124 and expanding it into the RAM 123 for execution.

[0056] The acquisition unit 131 acquires image data representing a captured image.

[0057] The analyzing unit 132 analyzes the image data acquired by the acquiring unit 131 .

[0058] The detection unit 133 detects the occurrence of a preset abnormality.

[0059] When the detection unit 133 detects that a preset abnormality has occurred, the inquiry unit 134 inquires about the registration information.

[0060] Based on the query result of the query unit 134, when an unauthorized object 80 that is not allowed to provide identification information of an identifiable individual is reflected in the image, the generation unit 135 generates additional information of the image data acquired by the acquisition unit 131 without including the identification information related to the unauthorized object 80.

[0061] The transmission unit 136 transmits the supplementary information generated by the generation unit 135 to the outside of the information processing system 100 via the communication mechanism 20 .

[0062] Thus, the flow of information processing performed by the information processing system 100 having such a hardware configuration and functional configuration is described in detail, but the information processing system 100 may store registration information in advance before the flow.

[0063] Figure 414 is a diagram showing an example of registration information stored in the information processing system 100 according to the present embodiment. The registration information includes a first item 141 , a second item 142 , a third item 143 , a fourth item 144 , a fifth item 145 , and a sixth item 146 .

[0064] The first item 141 is an item indicating a registration number. For example, different registration numbers may be assigned to the objects 80 in the order of registration.

[0065] The second item 142 is an item indicating the type of the object 80. If the object 80 is a person, it can be registered as "person" in this item. If the object 80 is a dog as a pet, it can be registered as "animal (dog)" in this item. If the object 80 is a robot as a property, it can be registered as "property (robot)" in this item.

[0066] The third item 143 is an item indicating a feature quantity for recognizing the object 80 from the image. The feature quantity can be extracted from the image showing the object 80. As an example, in the case where the object 80 is a person, the feature quantity can be data obtained by digitizing the contour of the face, the position, size, and color of the eyes, nose, mouth, and ears extracted from the face image.

[0067] The fourth item 144 is an item indicating the name of the object 80. When the object 80 is a person, the name of the person can be registered in this item. When the object 80 is a pet or a robot, the name given by the owner can be registered in this item.

[0068] The fifth item 145 is an item indicating the owner of the object 80. When the object 80 is a person, this item may be blank. Alternatively, when the object 80 is a minor, the name of the guardian may be registered in this item. When the object 80 is a pet or a robot, the name of the owner (the owner or the purchaser, etc.) may be registered in this item.

[0069] The sixth item 146 is an item indicating whether or not the identification information can be provided. The "permit" mark indicates that the identification information is permitted to be provided, that is, the identification information is permitted to be provided, and the "do not permit" mark indicates that the identification information is not permitted to be provided.

[0070] Whether such provision is permitted can be registered according to the degree of abnormality. In this figure, the case where the object 80 is shaken is set to "Emergency: Low", the case where the object 80 falls is set to "Emergency: Medium", and the case where the object 80 falls and does not move is set to "Emergency: High", and whether the provision is permitted is registered for each case. For example, regarding "Mr. X" with registration number 001, it is indicated that the provision of identification information is not permitted in the case of shaking or falling, and the provision of identification information is permitted only in the case of falling and not moving. On the other hand, regarding "Mr. Z" with registration number 003, it is indicated that the provision of identification information is permitted regardless of the case of shaking, falling, or falling and not moving.

[0071] In addition, in this figure, a fall is shown as an example as an abnormal situation, but it is not limited to this, and various abnormalities can be imagined. For example, an earthquake is imagined as an abnormality. In this case, the magnitude of the earthquake can be used as the degree of the abnormality, and the provision of registration can also be performed according to the magnitude of the earthquake. In addition, various abnormalities such as fire, crime, falling from a bed in a hospital or medical field, or changes in consciousness can also be imagined.

[0072] The information processing system 100 may store such registration information in advance in the ROM 122 or the storage device 124. In addition, it is also possible to store the registration information outside the information processing system 100, but in this case, it is necessary to send an image to the outside every time the information processing system 100 verifies the registration information, which may lead to leakage of personal information or privacy information. Therefore, for example, the registration information is preferably stored inside the information processing system 100.

[0073] In addition, even in such a case, the registration process of the registration information itself can be performed via the cloud or the like. That is, a user who wishes to register can upload a facial image or the like via the cloud or the like, and provide a registration name or whether or not to provide it. Then, feature quantities can be extracted in the cloud or the like. Furthermore, the information processing system 100 can obtain the registration information shown in this figure from the cloud or the like. By performing the registration process via the cloud or the like in this way, it is not necessary to extract feature quantities in the information processing system 100, and thus the processing load in the information processing system 100 can be reduced. Furthermore, since it is not necessary to store the image itself, such as the facial image, in the information processing system 100, the degree of tension in the storage capacity of the information processing system 100 can be suppressed.

[0074] Figure 5 1 is a diagram showing an example of the flow of information processing executed by the information processing system 100 according to the present embodiment. This flow is executed by the CPU 121 reading the information processing program from the ROM 122 or the storage device 124 and loading it into the RAM 123 for execution.

[0075] In step S151 , the CPU 121 as the acquisition unit 131 acquires image data representing an image of the space 60 captured by the camera 127 .

[0076] In step S152, the CPU 121 as the analysis unit 132 analyzes the image data acquired in step S151. For example, the CPU 121 analyzes the image data by AI (Artificial Intelligence) to detect objects including the target object 80. At this time, the CPU 121 can detect the type or position of the object in the image by using YOLO (You Only Look Once) or the like as an AI model.

[0077] Furthermore, the CPU 121 can acquire skeleton information by using HRNet (High Resolution Network) or the like as an AI model, and estimate the posture of the object 80 in the image or the position of the skeleton.

[0078] In step S153, the CPU 121 as the detection unit 133 determines whether a preset abnormality is detected. For example, the CPU 121 determines whether an abnormality is detected in the object 80 based on the analysis result in step S152. As an example, the CPU 121 detects that an abnormality has occurred in the object 80A when it is recognized that the object 80A has fallen based on the posture inference result in step S152. When an abnormality is detected in step S153 ("Yes"), the CPU 121 advances the processing to step S154.

[0079] In step S154, the CPU 121 as the inquiry unit 134 determines whether the provision of identification information is allowed. For example, in the case where the object 80A is identified to have fallen, the CPU 121 inquires about the registration information to confirm whether the object 80A is registered. At this time, it is assumed that the type of the object 80A is known to be a person through object detection. In this case, the CPU 121 first sets the second item 142 of the registration information to "person" and filters the registration information. Thus, the registration numbers 001, 002, and 003 are extracted, and the registration numbers 004 and 005 are excluded. Next, the CPU 121 checks the image of the object 80A with the feature quantity of the third item 143 in the registration numbers 001, 002, and 003. Thus, as a result of face authentication, the CPU 121 recognizes that the object 80A is "Mr. Z" of the registration number 003. In this way, the CPU 121 can reduce the time required for the recognition process and improve the recognition accuracy by filtering the registration information according to the type of the object 80 and then checking.

[0080] Furthermore, CPU 121 refers to item 146 of the sixth registration number 003 to determine whether identification information can be provided in the case of a fall. When it is confirmed that the column "Emergency: Medium (Fall)" is marked as "Allow", CPU 121 determines that it is allowed to provide identification information. That is, CPU 121 determines that object 80A is an allowed object 80 that is allowed to provide identification information in the case of a fall. When it is determined in step S154 that it is allowed to provide identification information ("Yes"), CPU 121 advances the processing to step S155.

[0081] In step S155, the CPU 121 generates the supplementary information of the image data acquired in step S151 including the identification information of the object 80A as the generating unit 135. That is, when a predetermined abnormality occurs and the permitted object 80 that is permitted to provide identification information is shown in the image, the CPU 121 generates the supplementary information including the identification information related to the permitted object 80.

[0082] Here, as an example, the "accompanying information" may be meta information of the image data. The meta information may be expressed in, for example, XML (Extensible Markup Language) or JSON (JavaScript (registered trademark) Object Notation) format.

[0083] The meta information may include object information related to objects other than the object 80 in the image. The object information related to objects other than the object 80 may include the name and coordinates of the object in the space 60 and recognition accuracy information indicating the accuracy of recognizing them from the image data.

[0084] Furthermore, the meta-information may include skeleton information related to the object 80 in the image. The skeleton information related to the object 80 may include the coordinates of the face, eyes, ears, shoulders, waist, hands, feet, etc. of the object 80, and recognition accuracy information indicating the accuracy of recognizing them from the image data. The individual of the object 80 cannot be recognized only by such skeleton information. Therefore, the CPU 121 generates the skeleton information related to the object 80 by including it in the supplementary information regardless of whether it can be provided.

[0085] Furthermore, the meta information may include identification information related to the object 80 in the image. The identification information related to the object 80 may include the name of the object 80, identification accuracy information indicating the accuracy of identifying the object 80 from the image data, and an image showing the object 80. Such a name of the object 80 or an image showing the object 80 can identify the individual of the object 80. Therefore, the CPU 121 generates the identification information by including the identification information in the supplementary information only when the provision of the identification information is permitted.

[0086] In addition, as a result of analyzing the image data in step S152, when a plurality of objects 80 are detected, the CPU 121 repeats the processing of steps S153 to S156 for the plurality of objects 80. When the object 80B is further detected in step S152, the CPU 121 determines in step S153 whether an abnormality is detected in the object 80B. As an example, based on the posture inference result in step S152, when it is recognized that the object 80B has not fallen and is not shaking, the CPU 121 determines that no abnormality is detected in the object 80B. When no abnormality is detected in step S153 ("No"), the CPU 121 causes the processing to proceed to step S156.

[0087] In step S156, the CPU 121 as the generating unit 135 generates the supplementary information of the image data acquired in step S151 without including the identification information of the object 80B. That is, when no abnormality occurs, the CPU 121 generates the supplementary information without including the identification information related to the unpermitted object 80 and the permitted object 80. In this way, when no abnormality is detected, the CPU 121 generates the supplementary information without including the identification information of the object 80, without determining whether to allow the provision of the identification information. In addition, the CPU 121 does not need to generate the supplementary information that has been generated twice.

[0088] When the object 80C is further detected in step S152, the CPU 121 determines in step S153 whether an abnormality is detected in the object 80C. For example, when the CPU 121 recognizes that the object 80C is shaking based on the posture estimation result in step S152, the abnormality is detected in the object 80C. When an abnormality is detected in step S153, the CPU 121 advances the process to step S154.

[0089] In step S154, when it is recognized that the object 80C is shaking, the CPU 121 queries the registration information to confirm whether the object 80C is registered. In addition, the CPU 121 recognizes that the object 80C is "Mr. X" of the registration number 001. In addition, the CPU 121 refers to the sixth item 146 of the registration number 001 to determine whether the identification information can be provided in the case of shaking. When it is confirmed that the "Urgency: Low (Shaking)" column is marked as "Not Allowed", the CPU 121 determines that the identification information is not allowed to be provided. That is, the CPU 121 determines that the object 80C is an unauthorized object 80 that is not allowed to provide identification information in the case of shaking. In addition, the CPU 121 can also determine that the object 80 that is not registered in the registration information is an unauthorized object 80. When it is determined in step S154 that the identification information is not allowed to be provided ("No"), the CPU 121 causes the processing to enter step S156.

[0090] In step S156, the CPU 121 generates the supplementary information of the image data acquired in step S151 without including the identification information of the object 80C. That is, when the unauthorized object 80, which is not allowed to provide the identification information of the identifiable individual, is shown in the image, the CPU 121 generates the supplementary information of the image data without including the identification information related to the unauthorized object 80. In this way, even if an abnormality is detected, the CPU 121 generates the supplementary information without including the identification information of the object 80 when the provision of the identification information is not allowed.

[0091] In step S157 , the CPU 121 as the transmission unit 136 transmits the supplementary information generated in steps S155 and S156 to the cloud system 40 outside the information processing system 100 via the communication mechanism 20 . Then, the information processing system 100 ends this flow.

[0092] Figure 6 An example of the supplementary information generated by the information processing system 100 according to the present embodiment is shown. In this figure, supplementary information is shown when the object 80A, the object 80B, and the object 80C, and the product rack m and the product rack n are reflected in the image.

[0093] Merchandise rack m and merchandise rack n are objects other than the object 80. Therefore, the supplementary information may include the name, coordinates, and recognition accuracy information of the merchandise racks as object information related to the merchandise racks m and merchandise racks n. In addition, the supplementary information may also include additional information such as the manufacturer, model, and size of the merchandise racks as object information.

[0094] The object 80A is an allowed object 80 that is allowed to provide identification information. Therefore, the incidental information may include the name of the object 80A, its identification accuracy information, and an image of the object 80A as identification information related to the object 80A. Furthermore, the incidental information may include the coordinates of the face, eyes, ears, shoulders, waist, hands, feet, etc. of the object 80A, and their identification accuracy information as skeleton information related to the object 80A.

[0095] Object 80B and object 80C are unauthorized objects 80 that are not allowed to provide identification information. Therefore, the accompanying information may include the coordinates of the face, eyes, ears, shoulders, waist, hands, feet, etc. of object 80B and object 80C, and their identification accuracy information as skeleton information of object 80B and object 80C.

[0096] In the past, network cameras were used as recording devices to confirm past images in the event of an emergency. In recent years, with the advancement of AI technologies such as deep learning, objects or people reflected in images can be recognized in real time and with high accuracy, and are being applied in various fields. Examples of applications include detection of crowded conditions in restaurants, unmanned checkout in stores, detection of intruders for prevention, and detection of falls or accidents.

[0097] In order to analyze images in real time, the dynamic images captured by the camera are sent to the cloud or a high-performance edge server as streaming information, and the AI ​​engine of the cloud or edge server performs the analysis. However, in recent years, AI cameras that have hardware accelerators on the camera side and perform image analysis on the camera side have become practical.

[0098] Advantages of performing AI processing on the camera side include protecting privacy and reducing network traffic. That is, in existing network cameras, when a moving image is analyzed using a cloud or edge server, the moving image required for analysis is sent via the network, so there is a risk that personal information or privacy information will leak out from the image being sent. On the other hand, in an endpoint AI camera that performs AI processing within the camera, image analysis is performed within the camera and sent to the outside as meta-information that excludes personal information or privacy information. In this way, since a structure that does not transmit personal information or privacy information on the network can be adopted, the risk of leakage of personal information or privacy information can be reduced. In addition, when image analysis is performed within the AI ​​camera, since there is no need to send moving images to the outside, the amount of communication can be extremely reduced.

[0099] However, in endpoint AI cameras that perform AI processing within the camera, only meta-information excluding personal information or privacy information is sent to the outside, so there is a problem of insufficient information and inability to respond flexibly when an emergency occurs. For example, in a camera that analyzes a mixed situation, the movement of people is analyzed based on image data, and information on the number or position of people in the space is sent to the outside as meta-information. In such a camera, if a person falls, the incident can be notified to the outside, but because personal information or privacy information is excluded, it is sometimes difficult to identify the person who fell and respond quickly.

[0100] To address this problem, there is a known method of including personal information or private information such as the name or image of the person who fell in an emergency in meta-information and transmitting it to the outside. Here, a camera that does not collect personal information or private information but only detects an emergency situation can be treated as a type of sensor. On the other hand, a camera that transmits personal information or private information to the outside in this way needs to comply with privacy guidelines like a general network camera, and is difficult to be installed in a public place such as a hospital or school.

[0101] Therefore, the information processing system 100 according to the present embodiment acquires image data, and when an unauthorized object 80 that is not allowed to provide identification information is shown in the image, the information processing system 100 generates the accompanying information of the image data without including the identification information related to the unauthorized object 80. Thus, according to the information processing system 100, it is possible to prevent the generation of information that can identify an individual for an unauthorized object.

[0102] In addition, the above embodiment can be transformed into various forms and can also be applied to various purposes. For example, in the above description, as an example, the CPU 121 uses image data to detect that an abnormality has occurred in the object 80. However, instead of, or on the basis of this, the CPU 121 can receive sensor data from a sensor 128 such as a pulse sensor, and use the sensor data to detect that an abnormality has occurred in the object 80. In addition, in the above description, as an example, it is shown that it is determined whether an abnormality has been detected for each object 80. However, instead of, or on the basis of this, the CPU 121 can determine whether an abnormality has been detected for each space 60. For example, the CPU 121 can receive sensor data from a sensor 128 such as a fire sensor or an earthquake sensor, and use the sensor data to detect that an abnormality has occurred in the space 60. In addition, the CPU 121 can also obtain voice or messages such as "a person fell", "an earthquake occurred" or "a fire occurred" via the user interface 126, and detect that an abnormality has occurred based on the voice or message.

[0103] Furthermore, in the above description, as an example, only the case where the CPU 121 sends the generated incidental information to the outside is shown. However, in addition to this, the CPU 121 can also display the incidental information sent to the outside on a preset display mechanism. As an example, the CPU 121 can display the incidental information sent to the outside on a monitor included in the user interface 126. By configuring such a monitor in the guard room or backyard of the store, etc., the incidental information can be visualized so as not to hinder the business of the store. Furthermore, in the case where a monitor is integrally provided with the camera 127, the CPU 121 can also display the incidental information sent to the outside on the monitor integrally provided with the camera 127. Thus, the incidental information can be visualized at the scene where the photography is actually performed. In addition, as the timing of sending the incidental information, the CPU 121 can display it every time the incidental information is sent to the outside, can display it in a preset cycle, can display it when an abnormality is detected, and can display it when a preset user input is performed.

[0104] Furthermore, in the above-mentioned embodiments, the processor refers to a processor in a broad sense, including a general-purpose processor (such as a CPU, etc.) or a dedicated processor (such as a GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0105] Furthermore, the actions of the processors in the above embodiments may be performed not only by one processor but also by a plurality of processors in physically separate locations. Furthermore, the order of the actions of the processors is not limited to the order described in the above embodiments and may be changed as appropriate.

[0106] Furthermore, as an example, the information processing system 100 in the present embodiment is described as being composed of a single device, but may be composed of a plurality of devices.

[0107] Furthermore, the processing performed by the information processing system 100 according to the above-mentioned embodiment may be processing performed by software, processing performed by hardware, or processing combined with both. Furthermore, the processing performed by each part of the information processing system 100 may be stored in a storage medium and circulated as a program.

[0108] Furthermore, the present invention is not limited to the above, and can of course be implemented with various modifications other than the above without departing from the gist of the present invention.

[0109] Regarding the above embodiment, the following supplementary notes are further disclosed. (((1)))

[0111] An information processing system comprises a processor,

[0112] The processor performs the following processing:

[0113] acquiring image data representing the captured image; and

[0114] When an unauthorized object that is not permitted to provide identification information that can identify an individual appears in the image, the supplementary information of the image data is generated without including the identification information related to the unauthorized object. (((2)))

[0116] The information processing system according to (((1))), wherein:

[0117] The processor generates the supplementary information including the identification information related to the permitted object when a permitted object that is permitted to provide the identification information is shown in the image. (((3)))

[0119] The information processing system according to (((2))), wherein:

[0120] The processor generates the supplementary information including the identification information related to the permitted object when a predetermined abnormality occurs and the permitted object appears in the image. (((4)))

[0122] The information processing system according to (((3))), wherein:

[0123] The processor detects that the abnormality has occurred using the image data. (((5)))

[0125] The information processing system according to (((3))) or (((4))), wherein:

[0126] The processor performs the following processing:

[0127] receiving sensor data; and

[0128] The occurrence of the abnormality is detected using the sensor data. (((6)))

[0130] The information processing system according to any one of (((3))) to (((5))), wherein:

[0131] The processor generates the supplementary information without including the identification information related to the non-permitted object and the permitted object when the abnormality does not occur. (((7)))

[0133] The information processing system according to any one of (((1))) to (((6))), wherein:

[0134] The processor transmits the supplementary information to the outside via a communication mechanism. (((8)))

[0136] The information processing system according to (((7))), wherein:

[0137] The processor displays the additional information transmitted to the outside on a preset display mechanism. (((9)))

[0139] The information processing system according to any one of (((1))) to (((8))), wherein:

[0140] The incidental information includes at least one of skeleton information related to the target object and object information related to an object other than the target object. (((10)))

[0142] The information processing system according to (((9))), wherein:

[0143] The skeleton information or the object information includes recognition accuracy information recognized from the image data. (((11)))

[0145] The information processing system according to any one of (((1))) to (((10))), wherein:

[0146] The identification information includes at least one of a name of the object and an image showing the object. (((12)))

[0148] An information processing program that causes a computer to perform the following processing:

[0149] acquiring image data representing the captured image; and

[0150] When an unauthorized object that is not permitted to provide identification information that can identify an individual appears in the image, the supplementary information of the image data is generated without including the identification information related to the unauthorized object.

[0151] According to (((1))), it is possible to provide an information processing system that can prevent the generation of information that can identify an individual for an unauthorized object.

[0152] According to (((2))), it is possible to prevent information that can identify an individual from being generated for an unauthorized object, and to generate information that can identify an individual for an authorized object.

[0153] According to (((3))), when information that can identify an individual is generated, the occurrence of a preset abnormality can be used as a condition.

[0154] According to (((4))), when it is detected that an abnormality has occurred, the required data can be reduced compared to the case where data other than image data is used.

[0155] According to (((5))), when the occurrence of an abnormality is detected, it is possible to detect the occurrence of various abnormalities compared to the case of using image data or the like.

[0156] According to (((6))), when no abnormality occurs, it is possible to prevent the generation of information that can identify an individual regardless of whether it is permitted.

[0157] According to (((7))), a service to which the incidental information is applied can be provided at the destination to which the incidental information is sent.

[0158] According to (((8))), it is possible to visualize what the incidental information sent is.

[0159] According to (((9))), information on the degree to which an individual cannot be identified with respect to an object or information on an object different from the object can be generated by including it in the supplementary information regardless of whether it is permitted or not.

[0160] According to (((10))), the degree of reliability of information recognized from image data can be informed.

[0161] According to (((11))), it is possible to prevent information with a high probability of being able to identify an individual from being generated for an unauthorized object.

[0162] According to (((12))), it is possible to provide an information processing program that can prevent the generation of information that can identify an individual for an unauthorized object.

[0163] The above-mentioned embodiments of the present invention are provided for the purpose of illustration and description. In addition, the embodiments of the present invention do not fully and exhaustively include the present invention, and do not limit the present invention to the disclosed methods. Obviously, various modifications and changes are self-evident to those skilled in the art to which the present invention belongs. The present embodiment is selected and described in order to most easily understand the principles of the present invention and its application. Thus, other technicians in the art can understand the present invention through various variations optimized for specific uses assumed to be various embodiments. The scope of the present invention is defined by the above claims and their equivalents.

[0164] Explanation of symbols

[0165] 10-information provision system, 20-communication mechanism, 40-cloud system, 60-space, 80-object, 100-information processing system, 121-CPU.

Claims

1. An information processing system comprising a processor, The processor performs the following processing: acquiring image data representing the captured image; and When an unauthorized object that is not permitted to provide identification information that can identify an individual appears in the image, the supplementary information of the image data is generated without including the identification information related to the unauthorized object.

2. The information processing system according to claim 1, wherein: The processor generates the supplementary information including the identification information related to the permitted object when a permitted object that is permitted to provide the identification information is shown in the image.

3. The information processing system according to claim 2, wherein: The processor generates the supplementary information including the identification information related to the permitted object when a predetermined abnormality occurs and the permitted object appears in the image.

4. The information processing system according to claim 3, wherein: The processor detects that the abnormality has occurred using the image data.

5. The information processing system according to claim 3 or 4, wherein: The processor performs the following processing: receiving sensor data; and The occurrence of the abnormality is detected using the sensor data.

6. The information processing system according to any one of claims 3 to 5, wherein: The processor generates the supplementary information without including the identification information related to the non-permitted object and the permitted object when the abnormality does not occur.

7. The information processing system according to any one of claims 1 to 6, wherein: The processor transmits the supplementary information to the outside via a communication mechanism.

8. The information processing system according to claim 7, wherein: The processor displays the additional information transmitted to the outside on a preset display mechanism.

9. The information processing system according to any one of claims 1 to 8, wherein: The incidental information includes at least one of skeleton information related to the target object and object information related to an object other than the target object.

10. The information processing system according to claim 9, wherein: The skeleton information or the object information includes recognition accuracy information recognized from the image data.

11. The information processing system according to any one of claims 1 to 10, wherein: The identification information includes at least one of a name of the object and an image showing the object.

12. A storage medium storing an information processing program for causing a computer to perform the following processing: acquiring image data representing the captured image; and When an unauthorized object that is not permitted to provide identification information that can identify an individual appears in the image, the supplementary information of the image data is generated without including the identification information related to the unauthorized object.

13. An information processing method comprising the following steps: acquiring image data representing the captured image; and When an unauthorized object that is not permitted to provide identification information that can identify an individual appears in the image, the supplementary information of the image data is generated without including the identification information related to the unauthorized object.

Images