Information processing system and information processing program

Abstract

To provide an information processing system and an information processing program capable of preventing generation of information that allows an individual to be identified regarding a not-permitted object.SOLUTION: An information processing system acquires image data indicating a photographed image, and when an unauthorized object in which provision of identification information for allowing an individual to be identified is not authorized is captured in the image, generates incidental information of the image data without containing the identification information regarding the unauthorized object.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an information processing system and an information processing program. [Background technology]

[0002] Patent document 1 proposes a video surveillance system comprising an imaging means for sending imaging conditions from a monitored location and capturing images, a recognition means for recognizing the human body based on the imaging information output by the imaging means, an abstraction means for abstracting information relating to the human body based on the output of the recognition means, a synthesis means for synthesizing the output of the abstraction means and the output of the imaging means, and an information transmission line for transmitting the output of the synthesis means to the monitored location.

[0003] Patent Document 2 proposes a location information system that includes a server having a function of determining the position of a mobile terminal or portable transmitter that performs short-range communication with a transmitter and / or receiver based on unique information of the transmitter or receiver received, in which the server includes an area setting means for setting a privacy protection area in any area of ​​a specified area within a management area, an area determination means for determining the position of the mobile terminal or portable transmitter that performs short-range communication with the transmitter and / or receiver based on the unique information of the transmitter or receiver and determining whether the determined position is within the privacy protection area or not, and an information transmission means for transmitting the determined position of the mobile terminal or portable transmitter and map information of the management area including the position, as well as information on the determination result by the area determination means.When the position of the mobile terminal or portable transmitter is determined to be within the privacy protection area under normal circumstances, the personal information of the holder of the mobile terminal or portable transmitter is made inaccessible, whereas when the position of the mobile terminal or portable transmitter is determined to be within the privacy protection area under emergency circumstances, the personal information of the holder of the mobile terminal or portable transmitter is made accessible. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2002-142214 A [Patent Document 2] JP 2023-006192 A Summary of the Invention [Problem to be solved by the invention]

[0005] In a system for acquiring information from images captured in a space, there is known an information system that allows personal information to be viewed in an emergency. In such an information system, the personal information of all people is viewable, so there is a problem that even the personal information of people who have not given permission can be viewed. This problem is not limited to information about people, but also applies to information about animals (e.g., pets) and possessions (e.g., robots).

[0006] Therefore, an object of the present disclosure is to provide an information processing system and an information processing program that can prevent the generation of information that can individually identify an object that is not permitted. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, an information processing system according to a first embodiment includes a processor, which acquires image data representing a captured image, and, when the image depicts an unauthorized object for which provision of identification information that can individually identify the object is not permitted, generates additional information for the image data without including the identification information regarding the unauthorized object.

[0008] In an information processing system according to a second aspect, in the information processing system according to the first aspect, when the image contains an authorized object for which the provision of identification information is permitted, the processor generates the auxiliary information including the identification information relating to the authorized object.

[0009] In an information processing system according to a third aspect, in the information processing system according to the second aspect, when a predetermined abnormality occurs and the permitted object is captured in the image, the processor generates the auxiliary information including the identification information regarding the permitted object.

[0010] In an information processing system according to a fourth aspect, in the information processing system according to the third aspect, the processor detects that the abnormality has occurred by using the image data.

[0011] In an information processing system according to a fifth aspect, in the information processing system according to the third aspect, the processor receives sensor data and detects the occurrence of the abnormality using the sensor data.

[0012] In an information processing system according to a sixth aspect, in the information processing system according to the third aspect, when the abnormality does not occur, the processor generates the auxiliary information without including the identification information regarding the unauthorized objects and the authorized objects.

[0013] In an information processing system according to a seventh aspect, in the information processing system according to the first aspect, the processor transmits the auxiliary information to an outside via a communication means.

[0014] In an information processing system according to an eighth aspect, in the information processing system according to the seventh aspect, the processor causes a predetermined display means to display the auxiliary information to be transmitted to an outside.

[0015] In an information processing system according to a ninth aspect, in the information processing system according to any one of the first to eighth aspects, the auxiliary information includes at least one of skeletal information about the target object or object information about an object other than the target object.

[0016] An information processing system according to a tenth aspect is the information processing system according to the ninth aspect, wherein the skeleton information or the object information includes recognition accuracy information recognized from the image data.

[0017] In an information processing system according to an eleventh aspect, in an information processing system according to any one of the first to eighth aspects, the identification information includes at least one of a name of the object or an image in which the object is depicted.

[0018] An information processing program according to a twelfth aspect causes a computer to acquire image data representing a captured image, and, if the image depicts an unauthorized object for which the provision of identification information that can individually identify the object is not permitted, generates additional information for the image data without including the identification information regarding the unauthorized object. Effect of the Invention

[0019] According to the information processing system according to the first aspect, it is possible to provide an information processing system that can prevent information that can identify an individual object from being generated regarding an unauthorized object.

[0020] According to the information processing system of the second aspect, it is possible to generate information that can identify individuals of permitted objects while preventing the generation of information that can identify individuals of unauthorized objects.

[0021] According to the information processing system of the third aspect, the generation of information capable of identifying an individual can be performed on the condition that a predetermined abnormality has occurred.

[0022] According to the information processing system of the fourth aspect, it is possible to reduce the amount of data required to detect the occurrence of an abnormality, compared to a case in which data other than image data is used.

[0023] According to the information processing system according to the fifth aspect, it is possible to detect the occurrence of a wide variety of abnormalities, as compared to a case in which image data or the like is used to detect the occurrence of an abnormality.

[0024] According to the information processing system of the sixth aspect, when no abnormality occurs, it is possible to prevent information that can identify an individual from being generated regardless of whether it is permitted or not.

[0025] According to the information processing system of the seventh aspect, a service utilizing the incidental information can be provided at the destination of the incidental information.

[0026] According to the information processing system of the eighth aspect, it is possible to visualize what kind of information the accompanying information to be transmitted is.

[0027] According to the information processing system of the ninth aspect, information about the target object that is not sufficient to identify the individual object, and information about objects other than the target object, can be generated and included in the auxiliary information regardless of whether it is permitted or not.

[0028] According to the information processing system of the tenth aspect, it is possible to inform the degree of accuracy of information recognized from image data.

[0029] According to the information processing system of the eleventh aspect, it is possible to prevent information that is likely to identify an individual object from being generated for an unauthorized object.

[0030] According to the information processing program of the twelfth aspect, it is possible to provide an information processing program that can prevent information that can individually identify an object from being generated for an unauthorized object. [Brief description of the drawings]

[0031] [Figure 1] 1 is a diagram showing an example of a schematic configuration of an information providing system 10 according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a hardware configuration of an information processing system 100 according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing system 100 according to the present embodiment. [Figure 4] 2 is a diagram showing an example of registration information stored in the information processing system 100 according to the present embodiment. FIG. [Diagram 5] FIG. 2 is a diagram showing an example of a flow of information processing executed by the information processing system 100 according to the present embodiment. [Figure 6] 2 shows an example of auxiliary information generated by the information processing system 100 according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Hereinafter, an example of an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensional ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0033] 1 is a diagram showing an example of a schematic configuration of an information providing system 10 according to the present embodiment. The information providing system 10 includes a communication unit 20, a cloud system 40, and an information processing system 100.

[0034] The communication means 20 connects a plurality of computers so that they can communicate with each other. In this figure, the communication means 20 connects between the cloud system 40 and the information processing system 100. The communication means 20 may be various networks such as a LAN, a WAN, the Internet, or an intranet. Furthermore, the communication means 20 may connect the plurality of computers wirelessly or by wire.

[0035] The cloud system 40 is a computer system that utilizes cloud computing. The cloud system 40 provides various services to users via the communication means 20. In this embodiment, the cloud system 40 acquires, from the information processing system 100, additional information generated from image data of a space 60 (described later), and provides various services to users using the additional information.

[0036] The space 60 is a space to be photographed. In this figure, the space 60 is a store, and a case is shown as an example in which there are a plurality of objects 80A, 80B, and 80C (collectively referred to as "objects 80" when there is no need to distinguish between them) that are people in the store. Note that the space 60 is not limited to a store, but may be any space in which an image can be photographed, such as an office, a factory, a hospital, or a medical site. Furthermore, the object 80 is not limited to a person, but may be any object such as an animal (e.g., a pet) or a possession (e.g., a robot) for which it may be problematic if information that can identify an individual is generated without permission.

[0037] The information processing system 100 acquires image data showing an image captured in the space 60, and generates supplementary information for the image data. Here, when generating supplementary information from image data of the space 60, there may be a problem that identification information capable of identifying an individual, such as personal information or privacy information, may be generated without permission. Therefore, the information processing system 100 according to this embodiment generates supplementary information without including identification information related to an unauthorized object 80 for which provision of identification information is not permitted. This will be described in detail.

[0038] 2 is a diagram showing an example of a hardware configuration of the information processing system 100 according to this embodiment. The information processing system 100 has a CPU 121, a ROM 122, a RAM 123, a storage 124, a communication interface 125, a user interface 126, a camera 127, and a sensor 128. These components are connected to each other via a bus 129 so as to be able to communicate with each other.

[0039] The CPU 121 is a central processing unit that executes various programs and controls each component. The ROM 122 stores various programs and various data. The RAM 123 temporarily stores programs or data as a working area. The storage 124 is composed of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including an operating system and various data.

[0040] In the information processing system 100 according to this embodiment, an information processing program is stored in the ROM 122 or the storage 124. The CPU 121 reads out a target program from the ROM 122 or the storage 124, and executes it using the RAM 123 as a working area. Then, the CPU 121 controls each component and executes various types of arithmetic processing according to the program.

[0041] The communication interface 125 is an interface for the information processing system 100 to communicate with other devices. The user interface 126 is an input / output interface for the information processing system 100 to exchange information with a user. The user interface 126 may include, for example, a keyboard, a mouse, and a microphone as input devices, and a monitor and a speaker as output devices.

[0042] The camera 127 captures an image of the space 60 using an imaging element such as a CCD or a CMOS. Note that the term "image" may be interpreted as including both still images and moving images. The camera 127 outputs image data representing the captured image of the space 60.

[0043] The sensor 128 measures various physical quantities. As an example, the sensor 128 may be a fire sensor, an earthquake sensor, a pulse sensor, etc. The sensor 128 outputs, as sensor data, measurement values ​​obtained by measuring the physical quantities, or calculation results obtained by performing a predetermined calculation on the measurement values.

[0044] In the above description, the case where the sensor 128 is provided inside the information processing system 100 has been described as an example, but the present invention is not limited to this. The sensor 128 may be externally attached to the outside of the information processing system 100. In this case, the sensor 128 may be communicatively connected to the information processing system 100 via the communication means 20.

[0045] 3 is a diagram showing an example of a functional configuration of the information processing system 100 according to this embodiment. The information processing system 100 includes an acquisition unit 131, an analysis unit 132, a detection unit 133, a query unit 134, a generation unit 135, and a transmission unit 136. These functional configurations are realized by the CPU 121 reading out an information processing program from the ROM 122 or the storage 124, expanding the program in the RAM 123, and executing the program.

[0046] The acquisition unit 131 acquires image data indicating a captured image.

[0047] The analysis unit 132 analyzes the image data acquired by the acquisition unit 131 .

[0048] The detection unit 133 detects the occurrence of a predetermined abnormality.

[0049] When the detection unit 133 detects that a predetermined abnormality has occurred, the inquiry unit 134 inquires about the registered information.

[0050] In response to the result of the inquiry by the inquiry unit 134, when an unauthorized object 80 for which provision of identification information that can individually identify the object is not permitted is depicted in the image, the generation unit 135 generates additional information for the image data acquired by the acquisition unit 131 without including identification information regarding the unauthorized object 80.

[0051] The transmitting unit 136 transmits the additional information generated by the generating unit 135 to the outside of the information processing system 100 via the communication means 20 .

[0052] From now on, a flow in which the information processing system 100 having such a hardware configuration and functional configuration executes information processing will be described in detail. Prior to this flow, the information processing system 100 may store registration information in advance.

[0053] 4 is a diagram showing an example of registration information stored in the information processing system 100 according to this embodiment. The registration information includes a first item 141, a second item 142, a third item 143, a fourth item 144, a fifth item 145, and a sixth item 146.

[0054] The first item 141 is an item indicating a registration number. Different registration numbers may be assigned to the objects 80, for example, in the order of registration.

[0055] The second item 142 is an item indicating the type of the object 80. If the object 80 is a person, the item may be registered as "person." If the object 80 is a pet dog, the item may be registered as "animal (dog)." If the object is a robot that is a possession, the item may be registered as "possession (robot)."

[0056] The third item 143 is an item indicating a feature amount for identifying the object 80 from an image. The feature amount may be extracted from an image in which the object 80 appears. As an example, when the object 80 is a person, the feature amount may be data obtained by digitizing the facial outline, the positions, sizes, colors, etc. of the eyes, nose, mouth, and ears extracted from a face image.

[0057] The fourth item 144 is an item indicating the name of the object 80. If the object 80 is a person, the individual's name may be registered in this item. If the object 80 is a pet or a robot, the name given to the object by its owner may be registered in this item.

[0058] The fifth item 145 is an item indicating the owner of the object 80. If the object 80 is a person, the item may be left blank. Alternatively, if the object 80 is a minor, the name of the guardian may be registered in the item. If the object 80 is a pet or a robot, the name of the owner (keeper, purchaser, etc.) may be registered in the item.

[0059] The sixth item 146 is an item indicating whether or not the provision of identification information is permitted. A "◯" mark indicates that the provision of identification information is permitted, i.e., that the provision of identification information is agreed to, and a "×" mark indicates that the provision of identification information is not permitted.

[0060] Such availability of information may be registered for each level of abnormality. In this figure, availability of information is registered for each case, with "urgency: low" when the object 80 stumbles, "urgency: medium" when the object 80 falls, and "urgency: high" when the object 80 falls and does not move. For example, for "Mr. X" with registration number 001, the provision of identification information is not permitted when the object 80 stumbles or falls, and the provision of identification information is permitted only when the object 80 falls and does not move. On the other hand, for "Mr. Z" with registration number 003, the provision of identification information is permitted in all cases of stumble, fall, and fall and does not move.

[0061] In addition, in this figure, a case where a fall or the like is assumed as an abnormality is shown as an example, but this is not limited thereto, and various abnormalities may be assumed. For example, an earthquake may be assumed as an abnormality. In this case, the seismic intensity may be used as the degree of abnormality, and the availability of information may be registered for each seismic intensity. In addition, various abnormalities may be assumed, such as a fire, a crime, a fall from a bed in a hospital or medical facility, or a change in consciousness state.

[0062] The information processing system 100 may store such registration information in advance in the ROM 122 or the storage 124. It is also possible to store the registration information outside the information processing system 100. In that case, however, the information processing system 100 needs to transmit an image to the outside every time it checks the registration information, which may lead to the leakage of personal information or privacy information. For this reason, it is preferable to store the registration information inside the information processing system 100.

[0063] Even in such a case, the registration process of the registration information itself may be executed via a cloud or the like. That is, a user who wishes to register may upload a face image or the like via a cloud or the like, and register a name and whether or not the information can be provided. Next, features may be extracted in the cloud or the like. Then, the information processing system 100 may acquire the registration information as shown in this figure from the cloud or the like. By executing the registration process via a cloud or the like in this way, it is not necessary to extract features in the information processing system 100, so that the processing load in the information processing system 100 can be reduced. In addition, it is not necessary to store the image itself, such as a face image, in the information processing system, so that the storage capacity of the information processing system 100 can be prevented from becoming congested.

[0064] 5 is a diagram showing an example of the flow of information processing executed by the information processing system 100 according to this embodiment. This flow is executed by the CPU 121 reading out an information processing program from the ROM 122 or the storage 124, expanding it in the RAM 123, and executing it.

[0065] In step S151 , the CPU 121 functions as the acquisition unit 131 to acquire image data indicating an image of the space 60 captured by the camera 127 .

[0066] In step S152, the CPU 121, functioning as the analysis unit 132, analyzes the image data acquired in step S151. For example, the CPU 121 analyzes the image data by AI (Artificial Intelligence) to detect objects including the target object 80. At this time, the CPU 121 may detect the type and position of the object in the image by using YOLO (You Only Look Once) or the like as an AI model.

[0067] Furthermore, the CPU 121 may estimate the posture and skeletal position of the object 80 in the image by acquiring skeletal information using a High Resolution Network (HRNet) or the like as an AI model.

[0068] In step S153, CPU 121, functioning as detection unit 133, determines whether or not a predetermined abnormality has been detected. For example, CPU 121 determines whether or not an abnormality has been detected in object 80 based on the result of the analysis in step S152. As an example, when CPU 121 recognizes that object 80A has fallen based on the result of the posture estimation in step S152, it detects that an abnormality has occurred in object 80A. If an abnormality is detected in step S153 (Yes), CPU 121 advances the process to step S154.

[0069] In step S154, the CPU 121, as the inquiry unit 134, determines whether or not the provision of identification information is permitted. For example, when the CPU 121 recognizes that the object 80A has fallen, it inquires about the registration information to confirm whether the object 80A is registered. At this time, it is assumed that the type of the object 80A is a person by object detection. In this case, the CPU 121 first sets the second item 142 of the registration information to "person" and filters the registration information. As a result, the registration numbers 001, 002, and 003 are extracted, and the registration numbers 004 and 005 are excluded. Next, the CPU 121 compares the image of the object 80A with the feature amount of the third item 143 in the registration numbers 001, 002, and 003. As a result of the face authentication, the CPU 121 identifies the object 80A as "Mr. Z" of the registration number 003. In this way, the CPU 121 can reduce the time required for the identification process and improve the accuracy of identification by filtering the registered information according to the type of the object 80 and then verifying it.

[0070] Then, CPU 121 refers to the sixth item 146 of registration number 003 and determines whether or not the provision of identification information is permitted in the event of a fall. When CPU 121 confirms that the "urgency: medium (fall)" column is marked with a "◯" mark, it determines that the provision of identification information is permitted. In other words, CPU 121 determines that object 80A is an authorized object 80 for which the provision of identification information is permitted in the event of a fall. If it is determined in step S154 that the provision of identification information is permitted (Yes), CPU 121 advances the process to step S155.

[0071] In step S155, CPU 121, as generation unit 135, generates supplementary information for the image data acquired in step S151, including the identification information of object 80A. That is, when a predetermined abnormality occurs and permitted object 80, for which provision of identification information is permitted, is captured in the image, CPU 121 generates supplementary information including identification information related to permitted object 80.

[0072] Here, the "accompanying information" may be, for example, meta information of the image data. The meta information may be expressed in, for example, XML (Extensible Markup Language) or JSON (JavaScript (registered trademark) Object Notation) format.

[0073] The meta information may include object information about objects other than the target object 80 in the image. The object information about objects other than the target object 80 may include the name of the object in the space 60, its coordinates, and recognition accuracy information indicating the accuracy with which the object was recognized from the image data.

[0074] The meta information may also include skeletal information about the object 80 in the image. The skeletal information about the object 80 may include the coordinates of the face, eyes, ears, shoulders, waist, hands, feet, etc. of the object 80, and recognition accuracy information indicating the accuracy with which the coordinates are recognized from the image data. Such skeletal information alone does not allow the individual object 80 to be identified. Therefore, the CPU 121 generates the skeletal information about the object 80 by including it in the supplementary information, regardless of whether it is available or not.

[0075] Furthermore, the meta information may include identification information regarding the object 80 in the image. The identification information regarding the object 80 may include the name of the object 80, recognition accuracy information indicating the accuracy with which the object 80 was recognized from the image data, and an image in which the object 80 appears. Such a name of the object 80 or an image in which the object 80 appears may identify the individual object 80. Therefore, only when provision of the identification information is permitted, the CPU 121 generates the identification information by including it in the incidental information.

[0076] If a plurality of objects 80 are detected as a result of analyzing the image data in step S152, CPU 121 repeatedly executes the processes of steps S153 to S156 for each of the plurality of objects 80. If object 80B is further detected in step S152, CPU 121 determines in step S153 whether or not an abnormality has been detected in object 80B. As an example, when CPU 121 recognizes that object 80B has not fallen or stumbled based on the result of posture estimation in step S152, it determines that no abnormality has been detected in object 80B. If no abnormality has been detected in step S153 (No), CPU 121 advances the process to step S156.

[0077] In step S156, CPU 121, as generation unit 135, generates incidental information of the image data acquired in step S151 without including identification information of object 80B. That is, when no abnormality occurs, CPU 121 generates incidental information without including identification information regarding unauthorized object 80 and permitted object 80. In this way, when no abnormality is detected, CPU 121 generates incidental information without including identification information of object 80, without even determining permission to provide identification information. Note that CPU 121 does not need to generate incidental information that has already been generated twice.

[0078] If the object 80C is further detected in step S152, in step S153, the CPU 121 determines whether or not an abnormality has been detected in the object 80C. As an example, when the CPU 121 recognizes that the object 80C has stumbled based on the result of the posture estimation in step S152, it detects that an abnormality has occurred in the object 80C. If an abnormality is detected in step S153, the CPU 121 advances the process to step S154.

[0079] In step S154, when the CPU 121 recognizes that the object 80C has stumbled, it refers to the registration information to confirm whether the object 80C is registered. Then, the CPU 121 identifies that the object 80C is "Mr. X" with the registration number 001. Then, the CPU 121 refers to the sixth item 146 of the registration number 001 to determine whether or not the identification information can be provided in the case of stumble. When the CPU 121 confirms that the "urgency: low (stumbled)" column is marked with an "X", it determines that the provision of the identification information is not permitted. That is, the CPU 121 determines that the object 80C is an unauthorized object 80 for which the provision of the identification information is not permitted in the case of stumble. The CPU 121 may also determine that an object 80 that is not registered in the registration information is an unauthorized object 80. If it is determined in step S154 that the provision of the identification information is not permitted (No), the CPU 121 advances the process to step S156.

[0080] In step S156, CPU 121 generates the incidental information of the image data acquired in step S151 without including the identification information of object 80C. That is, when an unauthorized object 80, for which provision of identification information capable of individually identifying the object 80 is not permitted, is captured in the image, CPU 121 generates the incidental information of the image data without including the identification information of the unauthorized object 80. In this way, even when an abnormality is detected, CPU 121 generates the incidental information without including the identification information of object 80 when provision of identification information is not permitted.

[0081] In step S157, the CPU 121, functioning as the transmission unit 136, transmits the auxiliary information generated in steps S155 and S156 to the cloud system 40, which is external to the information processing system 100, via the communication means 20. Then, the information processing system 100 ends this flow.

[0082] 6 shows an example of the incidental information generated by the information processing system 100 according to the present embodiment. In this figure, the incidental information is shown when an object 80A, an object 80B, an object 80C, a product shelf m, and a product shelf n are captured in an image.

[0083] The product shelf m and the product shelf n are objects other than the target object 80. Therefore, the incidental information may include the name of the product shelf, the coordinates, and their recognition accuracy information as object information regarding the product shelf m and the product shelf n. Note that the incidental information may further include additional information such as the manufacturer, model number, and dimensions of the product shelf as object information.

[0084] The object 80A is an authorized object 80 for which provision of identification information is permitted. Therefore, the incidental information may include, as identification information related to the object 80A, the name of the object 80A, its recognition accuracy information, and an image of the object 80A. In addition, the incidental information may include, as skeletal information related to the object 80A, the coordinates of the face, eyes, ears, shoulders, waist, hands, feet, etc. of the object 80A, and their recognition accuracy information.

[0085] The objects 80B and 80C are unauthorized objects 80 for which the provision of identification information is not permitted. Therefore, the supplementary information may include, as skeletal information of the objects 80B and 80C, the coordinates of the face, eyes, ears, shoulders, waist, hands, feet, etc. of the objects 80B and 80C, and their recognition accuracy information.

[0086] Traditionally, network cameras have been used as a means of recording in the event of an emergency, allowing users to check past images. Recently, advances in AI technology such as deep learning have made it possible to recognize objects and people captured in images in real time with high accuracy, and these cameras are being used in a variety of settings. Examples of use include detecting crowded conditions in restaurants, unmanned payment in stores, detecting intruders for crime prevention, and detecting falls and accidents.

[0087] To analyze images in real time, video images captured by a camera are sent as stream information to the cloud or a high-performance edge server, and then analyzed by the AI ​​engine of the cloud or edge server. In recent years, AI cameras that are equipped with a hardware accelerator on the camera side and perform image analysis on the camera side are beginning to be put into practical use.

[0088] Advantages of performing AI processing on the camera side include protecting privacy and reducing network traffic. That is, in conventional network cameras, when video images are analyzed on the cloud or edge server, the video images required for analysis are transmitted over the network, so there is a risk that personal information or private information may leak from the images being transmitted. On the other hand, endpoint AI cameras that perform AI processing within the camera analyze images within the camera and transmit them to the outside as meta information that excludes personal information and private information. In this way, a configuration can be adopted that does not transmit personal information or private information on the network, thereby reducing the risk of personal information or private information leaking. In addition, when performing image analysis within the AI ​​camera, there is no need to transmit video images to the outside, making it possible to drastically reduce communication volume.

[0089] However, endpoint AI cameras that perform AI processing within the camera only transmit meta information that excludes personal information and privacy information to the outside, which means that when an emergency occurs, there is a problem of insufficient information and the inability to respond flexibly. For example, a camera that analyzes crowding conditions analyzes people's movements from image data and transmits information on the number and positions of people in a space to the outside as meta information. When such a camera detects a person falling, it is possible to notify the outside of the event, but because personal information and privacy information are excluded, it may be difficult to identify the individual who fell and to respond quickly.

[0090] To address this issue, a method is known in which in the event of an emergency, personal and private information such as the name and image of the person who has fallen are included in the meta information and transmitted to the outside. While a camera that does not collect personal or private information and only detects emergencies can be treated as a type of sensor, a camera that transmits personal and private information in this way must follow privacy guidelines, just like a general network camera, making it difficult to install them in public places such as hospitals and schools.

[0091] Therefore, the information processing system 100 according to the present embodiment acquires image data, and when an unauthorized object 80 for which provision of identification information is not permitted is captured in the image, generates supplementary information for the image data without including identification information regarding the unauthorized object 80. In this way, the information processing system 100 can prevent information that can individually identify an unauthorized object from being generated.

[0092] The above-described embodiment may be modified in various ways or applied to various uses. For example, in the above description, a case where the CPU 121 detects that an abnormality has occurred in the object 80 using image data has been shown as an example. However, instead of or in addition to this, the CPU 121 may receive sensor data from a sensor 128 such as a pulse sensor, and use the sensor data to detect that an abnormality has occurred in the object 80. Also, in the above description, a case where whether or not an abnormality has been detected for each object 80 has been shown as an example. However, instead of or in addition to this, the CPU 121 may determine whether or not an abnormality has been detected for each space 60. For example, the CPU 121 may receive sensor data from a sensor 128 such as a fire sensor or an earthquake sensor, and use the sensor data to detect that an abnormality has occurred in the space 60. In addition, the CPU 121 may obtain a voice or message such as "a person has fallen," "an earthquake has occurred," or "a fire has occurred" via the user interface 126, and detect that an abnormality has occurred based on the voice or message.

[0093] In the above description, only the case where the CPU 121 transmits the generated incidental information to the outside has been shown as an example. However, in addition to this, the CPU 121 may display the incidental information to be transmitted to the outside on a predetermined display means. As an example, the CPU 121 may display the incidental information to be transmitted to the outside on a monitor included in the user interface 126. By arranging such a monitor in a security room or a backyard of the store, the incidental information can be visualized so as not to interfere with the business of the store. In addition, in the case where a monitor is provided integrally with the camera 127, the CPU 121 may display the incidental information to be transmitted to the outside on the monitor provided integrally with the camera 127. This allows the incidental information to be visualized at the site where the actual shooting is being performed. As for the timing of transmitting the incidental information, the CPU 121 may display the incidental information every time the incidental information is transmitted to the outside, may display it at a predetermined cycle, may display it when an abnormality is detected, or may display it when a predetermined user input is performed.

[0094] In addition, in the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0095] In addition, the operations of the processor in the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. The order of the operations of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate.

[0096] Furthermore, the information processing system 100 in the present embodiment has been described as being configured by a single device as an example, but may be configured by a plurality of devices.

[0097] The processes performed in the information processing system 100 according to the above embodiment may be processes performed by software, may be processes performed by hardware, or may be a combination of both. The processes performed in each part of the information processing system 100 may be stored as a program in a storage medium and distributed.

[0098] Furthermore, the present disclosure is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit and scope of the present disclosure.

[0099] The following supplementary notes are further disclosed regarding the above embodiment. (((1))) a processor, the processor comprising: acquiring image data indicative of the captured image; When the image includes an unauthorized object for which provision of identification information capable of identifying an individual is not permitted, supplementary information of the image data is generated without including the identification information regarding the unauthorized object. Information processing system.

[0100] (((2))) The processor, When the image includes an authorized object for which provision of the identification information is permitted, the auxiliary information is generated including the identification information related to the authorized object. An information processing system according to (((1))).

[0101] (((3))) The processor, generating the auxiliary information including the identification information relating to the permitted object when a predetermined abnormality occurs and the permitted object is captured in the image; The information processing system according to (((2))).

[0102] (((4))) The processor, detecting the occurrence of the anomaly using the image data; The information processing system according to (((3))).

[0103] (((5))) The processor, Accepts sensor data, Detecting the occurrence of the abnormality using the sensor data. The information processing system according to ((3))) or ((4)).

[0104] (((6))) The processor, When the abnormality does not occur, the supplementary information is generated without including the identification information regarding the unauthorized object and the authorized object. 13. An information processing system according to any one of ((3))) to (((5))).

[0105] (((7))) The processor, Transmitting the auxiliary information to an outside via a communication means. 1. An information processing system according to any one of ((1))) to (((6))).

[0106] (((8))) The processor, displaying the additional information to be transmitted to the outside on a predetermined display means; An information processing system according to (((7))).

[0107] (((9))) The auxiliary information includes at least one of skeleton information about the target object and object information about an object other than the target object. 1. An information processing system according to any one of ((1))) to (((8))).

[0108] (((10))) The skeleton information or the object information includes recognition accuracy information recognized from the image data. The information processing system according to (((9))).

[0109] (((11))) The identification information includes at least one of a name of the object or an image in which the object is captured. 1. An information processing system according to any one of ((1))) to (((10))).

[0110] (((12))) On the computer, Acquiring image data indicative of the captured image; When the image includes an unauthorized object for which provision of identification information capable of identifying an individual is not permitted, additional information of the image data is generated without including the identification information of the unauthorized object. Information processing program.

[0111] According to (((1))), it is possible to provide an information processing system that can prevent information that can identify an individual object from being generated for an unauthorized object.

[0112] According to (((2))), it is possible to prevent the generation of information that can identify individuals for unauthorized objects, while still generating information that can identify individuals for authorized objects.

[0113] According to (((3))), the occurrence of a predetermined abnormality can be set as a condition for generating information capable of identifying an individual.

[0114] According to (((4))), the amount of data required to detect the occurrence of an abnormality can be reduced compared to when data other than image data is used.

[0115] According to (((5))), when detecting the occurrence of an abnormality, it is possible to detect the occurrence of a wider variety of abnormalities than when image data or the like is used.

[0116] According to (((6))), when no abnormality has occurred, it is possible to prevent the generation of information that can identify an individual, regardless of whether or not such generation is permitted.

[0117] According to (((7))), a service that utilizes the incidental information can be provided at the destination of the incidental information.

[0118] According to (((8))), it is possible to visualize what kind of information is included in the additional information to be transmitted.

[0119] According to (((9))), information about the target object that does not allow the individual object to be identified, and information about objects other than the target object, can be generated and included in the auxiliary information regardless of whether it is permitted or not.

[0120] (((10))) makes it possible to know the degree of accuracy of the information recognized from the image data.

[0121] According to (((11))), it is possible to prevent the generation of information that is likely to be able to individually identify an object from being unauthorized.

[0122] According to (((12))), it is possible to provide an information processing program that can prevent information that can identify an individual object from being generated for an unauthorized object. [Explanation of symbols]

[0123] 10 Information provision system 20 Means of communication 40 Cloud System 60 space 80 Objects 100 Information Processing Systems 121 CPU

Claims

1. a processor, the processor comprising: acquiring image data indicative of the captured image; When the image includes an unauthorized object for which provision of identification information capable of identifying an individual is not permitted, supplementary information of the image data is generated without including the identification information regarding the unauthorized object. Information processing system.

2. The processor, When the image includes an authorized object for which provision of the identification information is permitted, the auxiliary information is generated including the identification information related to the authorized object. The information processing system according to claim 1 .

3. The processor, generating the auxiliary information including the identification information relating to the permitted object when a predetermined abnormality occurs and the permitted object is captured in the image; The information processing system according to claim 2 .

4. The processor, detecting the occurrence of the anomaly using the image data; The information processing system according to claim 3 .

5. The processor, Accepts sensor data, Detecting the occurrence of the abnormality using the sensor data. The information processing system according to claim 3 .

6. The processor, When the abnormality does not occur, the supplementary information is generated without including the identification information regarding the unauthorized object and the authorized object. The information processing system according to claim 3 .

7. The processor, Transmitting the auxiliary information to an outside via a communication means. The information processing system according to claim 1 .

8. The processor, displaying the additional information to be transmitted to the outside on a predetermined display means; The information processing system according to claim 7.

9. The auxiliary information includes at least one of skeleton information about the target object and object information about an object other than the target object. The information processing system according to any one of claims 1 to 8.

10. The skeleton information or the object information includes recognition accuracy information recognized from the image data. The information processing system according to claim 9.

11. The identification information includes at least one of a name of the object or an image in which the object is captured. The information processing system according to any one of claims 1 to 8.

12. On the computer, Acquiring image data indicative of the captured image; When the image includes an unauthorized object for which provision of identification information capable of identifying an individual is not permitted, additional information of the image data is generated without including the identification information of the unauthorized object. Information processing program.

Images