Information processing device, program, and information processing method

Abstract

An information processing device (P10) comprises: a trained model acquisition unit (P13) that, on the basis of feature amounts indicating relative positions between a plurality of parts of a person included in image information, acquires a trained model for classifying a person included in the image information; an image information acquisition unit (P11) that acquires image information including the person; a trained model computation unit (P12) that, on the basis of the trained model acquired by the trained model acquisition unit (P13) and first image information including a specific first person acquired by the image information acquisition unit (P11), carries out computations to adjust parameters of the trained model so as to classify whether the person included in the image information is the same person as the first person; and an output unit (P15) that outputs the computation result obtained by the trained model computation unit (P12) to an external device.

Description

Information Processing Apparatus, Program, and Information Processing Method

[0001] The present disclosure relates to an information processing apparatus, a program, and an information processing method.

[0002] Conventionally, a surveillance system including a plurality of cameras each having a function of performing video analysis using an AI model has been disclosed (see, for example, Patent Document 1). In the surveillance system described in Patent Document 1, an AI model for detecting specific behavior to detect suspicious behavior generated by an analysis device is transmitted to each camera via a communication network such as the Internet, and each camera is configured to detect suspicious behavior through video analysis using the AI model transmitted from the analysis device.

[0003] International Publication No. 2022 / 059123

[0004] However, when a trained model is distributed to a plurality of devices via an open communication network such as the Internet as in the surveillance system described in Patent Document 1, and a person is searched for by the plurality of devices, there is a problem that it is difficult to sufficiently protect the privacy of the person who is the search target because the trained model may be intercepted by a third party.

[0005] The present disclosure has been made in light of recognition of the above problem, and an object thereof is to provide an information processing apparatus, a program, and an information processing method that can protect the privacy of a person who is a search target.

[0006] The information processing apparatus according to the present disclosure includes: a trained model acquisition unit that acquires a trained model for classifying a person included in image information based on feature amounts indicating relative positions between a plurality of parts of the person included in the image information; an image information acquisition unit that acquires image information including a person; and a trained model calculation unit that performs calculation for adjusting parameters of the trained model so as to classify whether or not a person included in image information is the same person as a first specific person based on the trained model acquired by the trained model acquisition unit and first image information including the first specific person acquired by the image information acquisition unit; and an output unit that outputs a calculation result obtained by the trained model calculation unit to an external device.

[0007] According to this disclosure, the privacy of the person being investigated can be protected.

[0008] This is a block diagram showing the schematic configuration of an information processing system according to Embodiment 1. This is a block diagram showing an example of the hardware configuration of a first information processing device according to Embodiment 1. This is a block diagram showing an example of the hardware configuration of a second information processing device according to Embodiment 1. This is a flowchart showing an example of the process performed by the first information processing device according to Embodiment 1 for searching for a first person. This is a flowchart showing an example of the process performed by the first information processing device according to Embodiment 1 for searching for a second person. This is a block diagram showing the schematic configuration of an information processing system according to Embodiment 2. This is a block diagram showing the schematic configuration of an information processing system according to Embodiment 3. This is a flowchart showing an example of the process performed by the first information processing device according to Embodiment 3. This is a flowchart showing an example of the process performed by a model generation device according to Embodiment 3.

[0009] The embodiments of this disclosure will now be described in detail with reference to the drawings. Embodiment 1. First, the information processing system 1 according to Embodiment 1 will be described with reference to Figure 1. Figure 1 is a block diagram showing the schematic configuration of the information processing system 1 according to Embodiment 1. The information processing system 1 according to Embodiment 1 is a system for searching for a person by determining whether a person included in image information acquired using a plurality of devices is the same person as the person to be searched for. As shown in Figure 1, the information processing system 1 according to Embodiment 1 comprises a first information processing device P10, a second information processing device P20, a first image capture control device C10, and a second image capture control device C20, which are connected wirelessly or by wire via a communication network NT1 so that they can communicate with each other. For example, the communication network NT1 is configured as the Internet. Note that each component of the information processing system 1 may be connected to each other so that information can be communicated via devices not shown or other communication networks other than the communication network NT1. Each device constituting the information processing system 1 communicates with other devices using encrypted information.

[0010] Next, with reference to Figure 1, the first information processing device P10 according to Embodiment 1 will be described. The first information processing device P10 is a device for searching for a person who is the target of search by the user of the first information processing device P10 using each device that constitutes the information processing system 1, and for searching for a person who is the target of search by the user of the second information processing device P20. As shown in Figure 1, the first information processing device P10, as an information processing device, includes a first image information acquisition unit P11, a first trained model calculation unit P12, a first trained model acquisition unit P13, a first determination unit P14, a first output unit P15, a first storage control unit P16, and a first storage unit P17 that stores information used in various processes of the first information processing device P10. For example, the first information processing device P10 is composed of a smartphone, a tablet terminal, or other computer.

[0011] In Embodiment 1, an external device shown or not shown in Figure 1 that is connected to any of the devices constituting the information processing system 1 via the communication network NT 1 is also simply referred to as an "external device" with respect to that device. Furthermore, in Embodiment 1, a specific person who is the target of the information processing system 1's search and who is the target of the user of the first information processing device P10's search is also referred to as the "first person." For example, the first person is an acquaintance or family member whose whereabouts the user of the first information processing device wants to know. Similarly, in Embodiment 1, a specific person who is the target of the information processing system 1's search and who is the target of the user of the second information processing device P20's search is also referred to as the "second person."

[0012] The first pre-trained model acquisition unit P13, acting as a pre-trained model acquisition unit, acquires a pre-trained model for classifying people included in image information based on feature quantities indicating the relative positions between multiple parts of a person included in the image information. For example, the first pre-trained model acquisition unit P13 acquires a pre-trained model that has been pre-trained based on input of multiple image information generated by imaging people, each received from an external device via the communication network NT1, and calculates the probability that a person included in the image information belongs to a specific person class based on feature quantities indicating the relative positions between multiple parts of a person included in the input image information. For example, the first pre-trained model acquisition unit P13 acquires a pre-trained model from a device (not shown) that generates pre-trained models, or from a database (not shown) that stores pre-trained models. For example, the pre-trained model acquired by the first pre-trained model acquisition unit P13 is generated by deep learning. However, the pre-trained model acquired by the first pre-trained model acquisition unit P13 may also be generated by algorithms other than deep learning, such as regression, decision tree learning, Bayesian methods, or clustering.

[0013] For example, the first trained model acquisition unit P13 acquires a trained model for classifying people included in image information based on feature quantities that indicate the relative positions between three or more specific parts of a person's face and specific joints of their body included in the image information. For example, specific parts of a person's face included in the image information include the pupil, outer corner of the eye, inner corner of the eye, space between the eyebrows, tip of the nose, chin, and top of the head. Also, for example, specific joints of the body included in the image information include the shoulder, elbow, wrist, waist, hip, knee, and ankle. Since the relative positions between these specific parts of a person's face and specific joints of their body differ from person to person, it is possible to classify whether multiple people included in separately acquired image information are the same person or not based on feature quantities that indicate the relative positions between three or more of these parts.

[0014] Furthermore, for example, the first trained model acquisition unit P13 acquires the results of calculations performed by an external device via the communication network NT1 from an external device to adjust the parameters of a trained model so that the person included in the image information is classified as the same person as a specific person, based on feature quantities indicating the relative positions between multiple parts of the person included in the image information. For example, the first trained model acquisition unit P13 acquires the parameters of a trained model in which the parameters of a general-purpose trained model have been adjusted to suit a specific person, based on feature quantities indicating the relative positions between multiple parts of the person included in the image information, based on the communication network NT1 from an external device. Specifically, the first trained model acquisition unit P13 acquires the parameters of a second trained model, which is a trained model in which the parameters of a general-purpose trained model have been adjusted to suit a second person, based on feature quantities indicating the relative positions between multiple parts of the person included in the image information, based on the communication network NT1 from a second information processing device P20 as an external device.

[0015] Furthermore, for example, the first trained model acquisition unit P13 acquires a trained model from an external device via the communication network NT1, in which the parameters of a general-purpose trained model have been adjusted to suit a specific person, so as to classify whether the person included in the image information is the same person as a specific person, based on feature quantities indicating the relative positions between multiple parts of the person included in the image information. Specifically, the first trained model acquisition unit P13 acquires a second trained model from the second information processing device P20, which is an external device, via the communication network NT1, in which the parameters of a general-purpose trained model have been adjusted to suit a second person, so as to classify whether the person included in the image information is the same person as a second person, based on feature quantities indicating the relative positions between multiple parts of the person included in the image information. Note that the first trained model acquisition unit P13 constitutes the calculation result acquisition unit in Embodiment 1.

[0016] Generally, when a trained model is used to classify people in image information and determine whether a person in the image information is the same person as a specific person, the accuracy of the determination can be improved by adjusting the parameters of the trained model based on the characteristics of the specific person's appearance. In the information processing system according to Embodiment 1, the parameters of the trained model are adjusted based on feature quantities that indicate the relative positions between multiple parts of a specific person. For example, in a trained model for determining whether a person in image information is the same person as a specific person based on feature quantities that indicate the distance between the outer corner and inner corner of the eye, and the relative positions between the two pupils and the tip of the nose, if the distance between the outer corner and inner corner of the eye deviates more from the average value than the relative position between the two pupils and the tip of the nose, the parameters of the trained model are adjusted by assigning a greater weight to the distance between the outer corner and inner corner of the eye than to the relative position between the two pupils and the tip of the nose.

[0017] Furthermore, for example, the first trained model acquisition unit P13 acquires a trained model or the parameters of a trained model from an external device (not shown) via a communication network NT1. Furthermore, for example, the first trained model acquisition unit P13 acquires a trained model or the parameters of a trained model from a second information processing device P20, which is an external device, via a communication network NT1. Furthermore, the first trained model acquisition unit P13 acquires a trained model or the parameters of a trained model from a second information processing device P20, which is an external device, via a communication network NT1.

[0018] The first image information acquisition unit P11, acting as an image information acquisition unit, acquires image information including people. In other words, the first image information acquisition unit P11 acquires image information which is information of an image generated by imaging a region including people. For example, the first image information acquisition unit P11 acquires image information including people from an external device via the communication network NT1. Alternatively, for example, the first image information acquisition unit P11 refers to information stored in the first storage unit P17 and acquires image information including people from the information stored in the first storage unit P17. Alternatively, for example, the first image information acquisition unit P11 acquires image information including people that was generated when imaging a region including people was performed by an imaging device (not shown) provided in the first information processing device P10, which acts as a smartphone. The imaging device provided in the first information processing device P10 has an image sensor composed of, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and generates image information by converting incident light into an electrical signal.

[0019] The first trained model calculation unit P12, acting as a trained model calculation unit, performs calculations to adjust the parameters of the trained model acquired by the first trained model acquisition unit P13, based on the trained model acquired by the first trained model acquisition unit P13, so as to classify whether the person included in the image information is the same person as a specific person. For example, the first trained model calculation unit P12 performs calculations to adjust the parameters of the trained model acquired by the first trained model acquisition unit P13, based on the trained model acquired by the first trained model acquisition unit P13 and the first image information including the first person, which is a specific person, acquired by the first image information acquisition unit P11, so as to calculate the probability that the person included in the image information is the first person. In other words, the first trained model calculation unit P12 generates a first trained model with adjusted parameters based on the trained model acquired by the first trained model acquisition unit P13 and the first image information including a specific person, the first person, acquired by the first image information acquisition unit P11, so as to whether the person included in the image information is the same person as the first person.

[0020] Furthermore, for example, the first trained model calculation unit P12 generates a trained model with adjusted parameters based on the second trained model acquired by the first trained model acquisition unit P13 and the first image information including a specific person, the first person, acquired by the first image information acquisition unit P11. The trained unit P12 classifies whether the person included in the image information is the same person as the first person, and also classifies whether the person included in the image information is the same person as the second person.

[0021] Furthermore, for example, the first trained model calculation unit P12 generates a first trained model for classifying whether a person included in image information is the same person as a specific person related to the parameters, based on the trained model acquired by the first trained model acquisition unit P13 and the parameters acquired by the first trained model acquisition unit P13. Specifically, the first trained model calculation unit P12 generates a second trained model for classifying whether a person included in image information is the same person as a second person who is a specific person related to the parameters, based on the trained model acquired by the first trained model acquisition unit P13 and the parameters acquired by the first trained model acquisition unit P13.

[0022] Furthermore, it is desirable that the first trained model calculation unit P12 performs calculations to adjust the parameters of the trained model based on multiple image information, each including the first person. For example, the multiple image information, each including the first person, consists of multiple image information generated by capturing the first person at different times, multiple image information as frames constituting a moving image generated by capturing the first person, multiple image information generated by transforming the brightness, saturation, hue, orientation, and aspect ratio of the image information generated by capturing the first person using a specific algorithm, or a combination thereof. The first trained model calculation unit P12 may also have a function to perform the image information transformations described above.

[0023] The first determination unit P14, acting as a determination unit, determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person, based on the information acquired by the first trained model acquisition unit P13 and the image information acquired by the first image information acquisition unit P11. For example, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person that the trained model is searching for, based on the trained model acquired by the first trained model acquisition unit P13 and the image information acquired by the first image information acquisition unit P11. In other words, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person that the trained model is searching for, based on the output of the trained model acquired by inputting the image information acquired by the first image information acquisition unit P11 to the trained model acquired by the first trained model acquisition unit P13.

[0024] Furthermore, for example, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person that the trained model is searching for, based on the trained model generated based on the parameters acquired by the first trained model acquisition unit P13 and the image information acquired by the first image information acquisition unit P11. In other words, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person that the trained model is searching for, based on the output of the trained model obtained by inputting the image information acquired by the first image information acquisition unit P11 to the trained model generated based on the parameters acquired by the first trained model acquisition unit P13.

[0025] Specifically, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the first person, using the first trained model generated based on the trained model acquired by the first trained model acquisition unit P13 and the image information acquired by the first image information acquisition unit P11. Furthermore, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person, using the second trained model generated based on the information acquired by the first trained model acquisition unit P13 and the parameters acquired by the first trained model acquisition unit P13. Furthermore, the first determination unit P14 determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person, using the second trained model acquired by the first trained model acquisition unit P13.

[0026] For example, the first determination unit P14 inputs image information acquired by the first image information acquisition unit P11 to the first trained model to obtain the probability from the first trained model that the person in the image information is the first person, and determines that the person in the image information is the first person if the obtained probability is higher than a preset threshold. Also, for example, the first determination unit P14 inputs image information acquired by the first image information acquisition unit P11 to the second trained model to obtain the probability from the second trained model that the person in the image information is the second person, and determines that the person in the image information is the second person if the obtained probability is higher than a preset threshold.

[0027] Furthermore, the first determination unit P14 is not limited to determining whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specified person, in two ways: "is the specified person" and "is not the specified person." For example, the first determination unit P14 may be configured to determine whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specified person in three stages: high, medium, and low, indicating the likelihood that the person is the specified person, or it may be configured to determine in four or more stages.

[0028] Furthermore, the first determination unit P14 generates information related to the determination result based on the determination result of whether or not the person included in the image information acquired by the first image information acquisition unit P11 is the same person as a specific person. For example, the first determination unit P14 acquires determination information by generating determination information including the determination result based on the determination result of whether or not the person included in the image information acquired by the first image information acquisition unit P11 is the same person as a specific person. For example, the first determination unit P14 generates determination information including the determination result based on the determination result of whether or not the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the first person. For example, the first determination unit P14 generates information as determination information indicating a determination result that shows the person included in the image information acquired by the first image information acquisition unit P11 was not the first person. Also, for example, the first determination unit P14 generates information as determination information indicating a determination result that shows the person included in the image information acquired by the first image information acquisition unit P11 is the first person.

[0029] Furthermore, for example, the first determination unit P14 generates determination information including the determination result based on the determination result of whether or not the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person. For example, the first determination unit P14 generates information as determination information indicating a determination result that shows the person included in the image information acquired by the first image information acquisition unit P11 was not the second person. Also, for example, the first determination unit P14 generates determination information indicating a determination result that shows the person included in the image information acquired by the first image information acquisition unit P11 is the second person, and information regarding the location of the second person acquired based on the information contained in the image information. Specifically, the first determination unit P14 generates determination information including a determination result that shows the person included in the image information acquired by the first image information acquisition unit P11 is the second person, and information regarding a location identified based on the image included in the image information, the latitude and longitude generated by the image acquisition of the image information, the address or building name corresponding to these latitude and longitude, and the date and time the image information was generated by the image acquisition. Furthermore, for example, the first determination unit P14 generates determination information, including the determination result and information such as a telephone number, email address, and web page address indicating where to inquire about the location of the second person.

[0030] Furthermore, for example, the first determination unit P14 obtains information regarding the determination result from an external device that determines whether or not a person included in other image information is the same person as the first person, based on the calculation result by the first trained model calculation unit P12 and other image information other than the first image information. For example, the first determination unit P14 obtains determination information including the determination result of whether or not a person included in other image information is the same person as the first person from an external device that determines whether or not a person included in other image information is the same person as the first person, based on the calculation result by the first trained model calculation unit P12 and other image information other than the first image information. Specifically, the first determination unit P14 obtains determination information from the first imaging control device C10, which is an external device that has acquired the first trained model as a calculation result by the first trained model calculation unit P12. This determination information includes the determination result of whether the person included in the image information is the same person as the first person, based on the first trained model and the image information acquired by the first imaging control device C10.

[0031] For example, the first determination unit P14 acquires, as determination information, information indicating a determination result that the person included in the image information acquired by the external device was not the first person. Alternatively, as determination information, the first determination unit P14 acquires, as determination information, a determination result that the person included in the image information acquired by the external device is the first person, and information regarding the location of the first person acquired based on the information contained in the image information. Specifically, the first determination unit P14 acquires, as determination information, a determination result that the person included in the image information acquired by the first image information acquisition unit P11 is the first person, and information regarding the location identified based on the image contained in the image information, the latitude and longitude generated by the image acquisition of the image information, the address or building name corresponding to these latitude and longitude, and the date and time the image information was generated by the image acquisition. Alternatively, as determination information, the first determination unit P14 acquires, as determination information, the determination result, and information such as a telephone number, email address, and web page address indicating where to inquire about the location of the first person.

[0032] Furthermore, the first determination unit P14 acquires information regarding the determination result from an external device that determines whether the person included in the image information is the same person as the second person. For example, the first determination unit P14 acquires termination information from an external device that determines whether the person included in the image information is the same person as the second person based on the second trained model and the image information, based on the determination that the person included in the image information is the same person as the second person. This termination information is output by each device constituting the information processing system 1. Note that the first determination unit P14 constitutes the determination information acquisition unit in Embodiment 1.

[0033] Furthermore, the first determination unit P14 generates termination information in each device constituting the information processing system 1, indicating that the determination of whether the person included in the image information is the same person as the first person is terminated, based on the determination result indicating that the person included in the image information acquired by the first image information acquisition unit P11 is a specific person. For example, the first determination unit P14 generates termination information in each device constituting the information processing system 1, indicating that the determination of whether the person included in the image information is the same person as the first person is terminated, based on the determination result indicating that the person included in the image information acquired by the first image information acquisition unit P11 is a specific person, and the acquisition of a signal indicating that the user of the first information processing device P10 has performed an input operation to terminate the search to an input device (not shown) which is communicably connected to the first information processing device P10. For example, the user of the first information processing device P10 performs an input operation to terminate the search to the input device when the location of the first person is confirmed, based on the determination information generated by the first determination unit P14 or the information regarding the location of the first person included in the determination information acquired from an external device.

[0034] The first output unit P15, acting as an output unit, outputs various types of information to at least one of an external device connected to the first information processing device P10 via the communication network NT1 and a device (not shown) provided by the first information processing device P10. For example, the first output unit P15 outputs the calculation results of the first trained model calculation unit P12 to the external device. Specifically, the first output unit P15 outputs the first trained model generated by the first trained model calculation unit P12 to the external device as a calculation result of the first trained model calculation unit P12. Also specifically, the first output unit P15 outputs the parameters of the first trained model generated by the first trained model calculation unit P12 to the external device as a calculation result of the first trained model calculation unit P12.

[0035] Furthermore, for example, the first output unit P15 outputs the determination information generated by the first determination unit P14 to the second information processing device P20, which is an external device. Furthermore, for example, the first output unit P15 outputs the determination information generated by the first determination unit P14 to a notification device (not shown) which is an external device for notifying the user of the information processing system 1. Furthermore, for example, the first output unit P15 outputs the determination information generated by the first determination unit P14 to a notification device (not shown) provided in the first information processing device P10 for notifying the user of the first information processing device P10.

[0036] For example, the notification device as an external device and the notification device provided by the first information processing device P10 are composed of a display device such as a liquid crystal display that displays information as images, a sound notification device such as a speaker that outputs information as sound, or a combination thereof, and notify the user of the information processing system 1 of the determination information from the first output unit P15. For example, the notification device provided by the first information processing device P10 is composed of a display device that displays information as images, a sound notification device that outputs information as sound, or a combination thereof, provided by the first information processing device P10 which is composed of a smartphone, and notifies the user of the first information processing device P10 of the determination information from the first output unit P15. The first output unit P15 may be configured to output the entire determination information including the determination result from the first determination unit P14, or it may be configured to output only the determination result of whether or not the person included in the image information is the same person as the first person, based on the determination information including the determination result from the first determination unit P14.

[0037] Furthermore, for example, the first output unit P15 outputs information acquired from an external device to a notification device provided in the first information processing device P10. Specifically, the first output unit P15 outputs the judgment information acquired from the external device to the notification device provided in the first information processing device P10, thereby causing the notification device to notify the user of the first information processing device P10 of the judgment information acquired from the external device.

[0038] Furthermore, for example, the first output unit P15, based on the determination that the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person, outputs termination information to an external device indicating that each device constituting the information processing system 1 has finished determining whether or not the person included in the image information is the same person as the second person.

[0039] The first memory control unit P16, acting as a memory control unit, controls the storage of information by the first storage unit P17. For example, the first memory control unit P16 causes the first storage unit P17 to store information acquired from an external device by the first trained model acquisition unit P13. Specifically, the first memory control unit P16 causes the first storage unit P17 to store a trained model acquired from an external device by the first trained model acquisition unit P13. Also specifically, the first memory control unit P16 causes the first storage unit P17 to store the parameters of a trained model acquired from an external device by the first trained model acquisition unit P13.

[0040] Furthermore, the first memory control unit P16 deletes information about the specific person stored in the first memory unit P17, based on the determination result by the first determination unit P14 which indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the specific person being searched for. In other words, the first memory control unit P16 deletes information about the calculation results for adjusting parameters in conjunction with the specific person by an external device, stored in the first memory unit P17, based on the determination result by the first determination unit P14 which indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the specific person being searched for.

[0041] For example, if the determination result made by the first determination unit P14 using the second trained model indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the second person, the first memory control unit P16 deletes the information related to the generation of the second trained model of an external device, which was acquired by the first trained model acquisition unit P13 and is stored in the first memory unit P17. Specifically, if the determination result made by the first determination unit P14 using the second trained model indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the second person, the first memory control unit P16 deletes the second trained model, which is the calculation result of the second information processing device P20 acquired by the first trained model acquisition unit P13 and is stored in the first memory unit P17.

[0042] Specifically, if the determination result made by the first determination unit P14 using the second trained model indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the second person, the first memory control unit P16 deletes the parameters of the second trained model, which are the calculation results of the second information processing device P20 acquired by the first trained model acquisition unit P13 and stored in the first memory unit P17. Furthermore, if the determination result made by the first determination unit P14 using the second trained model indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the second person, the first memory control unit P16 may be configured to delete the trained model in its state before the parameters were adjusted to match the second person, or it may be configured not to delete it, or it may be configured to delete the first trained model or information regarding the parameters of the first trained model when the search for the first person is completed.

[0043] Furthermore, the first memory control unit P16 may be configured to delete information about the specific person stored in the first memory unit P17 when termination information is generated by the first determination unit P14, or when termination information is acquired from an external device by the first determination unit P14. Based on the determination result by the first determination unit P14, which indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the specific person being searched for, the first memory control unit P16 deletes information about the specific person stored in the first memory unit P17, thereby preventing information about the person being searched for from continuing to be stored in the first information processing device P10 after the completion of the search, and thereby protecting the privacy of the person being searched for.

[0044] Configured in this way, the first information processing apparatus P10 causes each apparatus configuring the information processing system 1 to search for a first person who is a search target by the user of the first information processing apparatus P10, and searches for a second person who is a search target by the user of the second information processing apparatus P20. For example, the first information processing apparatus P10 generates a first trained model based on image information including the first person acquired by the first image information acquisition unit P11, and outputs the first trained model or parameters of the first trained model to each apparatus configuring the information processing system 1. Thus, the first information processing apparatus P10 searches for the first person by using the first trained model to determine whether a person included in image information acquired by the first information processing apparatus P10 and each said apparatus is the same person as the first person. Further, for example, the first information processing apparatus P10 acquires a second trained model from the second information processing apparatus P20, or generates a second trained model based on parameters acquired from the second information processing apparatus P20, and searches for the second person by using the second trained model to determine whether a person included in image information acquired by the first information processing apparatus P10 is the same person as the second person.

[0045] Next, the second information processing apparatus P20 according to the first embodiment will be described with reference to FIG. 1. As shown in FIG. 1, the second information processing apparatus P20 serving as an information processing apparatus includes a second image information acquisition unit P21, a second trained model calculation unit P22, a second trained model acquisition unit P23, a second determination unit P24, a second output unit P25, a second storage control unit P26, and a second storage unit P27. The second information processing apparatus P20 is an apparatus that causes each apparatus configuring the information processing system 1 to search for a second person who is a search target by the user of the second information processing apparatus P20, and searches for a first person who is a search target by the user of the first information processing apparatus P10.

[0046] For example, the second information processing device P20 generates a second trained model based on image information including a second person acquired by the second image information acquisition unit P21, and outputs the second trained model or the parameters of the second trained model to each device constituting the information processing system 1. The second trained model then determines whether the person included in the image information acquired by the second information processing device P20 and each device is the same person as the second person, thereby performing a search for the second person. Alternatively, for example, the second information processing device P20 acquires a first trained model from the first information processing device P10, or generates a first trained model based on parameters acquired from the first information processing device P10, and the first trained model then determines whether the person included in the image information acquired by the second information processing device P20 is the same person as the first person, thereby performing a search for the first person. For example, the second information processing device P20 is composed of a smartphone, a tablet terminal, or other computer.

[0047] Furthermore, the functions of the second image information acquisition unit P21, the second trained model calculation unit P22, the second trained model acquisition unit P23, the second determination unit P24, the second output unit P25, the second memory control unit P26, and the second memory unit P27, which are provided in the second information processing device P20, are the same as those of the first image information acquisition unit P11, the first trained model calculation unit P12, the first trained model acquisition unit P13, the first determination unit P14, the first output unit P15, the first memory control unit P16, and the first memory unit P17, which are provided in the first information processing device P10, so their explanation will be omitted.

[0048] Next, a first imaging control apparatus C10 and a second imaging control apparatus C20 according to the first embodiment will be described with reference to FIG. 1. As shown in FIG. 1, each of the first imaging control apparatus C10 and the second imaging control apparatus C20 includes an image information acquisition unit C11, a trained model acquisition unit C13, a determination unit C14, an output unit C15, a storage control unit C16, and a storage unit C17. The first imaging control apparatus C10 and the second imaging control apparatus C20 are apparatuses for searching for a first person and a second person, respectively. For example, each of the first imaging control apparatus C10 and the second imaging control apparatus C20 acquires a first trained model from a first information processing apparatus P10, and searches for the first person by using the first trained model to determine whether or not a person included in the image information acquired by the image information acquisition unit is the same person as the first person. Further, for example, each of the first imaging control apparatus C10 and the second imaging control apparatus C20 acquires a second trained model from a second information processing apparatus P20, and searches for the second person by using the second trained model to determine whether or not a person included in the image information acquired by the image information acquisition unit is the same person as the second person.

[0049] For example, each of the first imaging control apparatus C10 and the second imaging control apparatus C20 is an apparatus for acquiring image information from an unillustrated imaging apparatus such as a monitoring camera installed outdoors or indoors to capture an image of the surroundings of the installation location, a camera provided in a moving body such as a vehicle or a drone, or another camera. The first imaging control apparatus C10 and the second imaging control apparatus C20 may be configured integrally with the imaging apparatus, may be communicably connected to the imaging apparatus via a communication network NT1, or may be configured to acquire image information from two or more imaging apparatuses different from each other, respectively.

[0050] The functions of the image information acquisition unit C11, the trained model acquisition unit C13, the determination unit C14, the output unit C15, the memory control unit C16, and the memory unit C17 provided in the first image processing device C10 and the second image processing device C20 are the same as those of the first image information acquisition unit P11, the first trained model calculation unit P12, the first trained model acquisition unit P13, the first determination unit P14, the first output unit P15, the first memory control unit P16, and the first memory unit P17 provided in the first information processing device P10, respectively, so their explanation is omitted. Furthermore, the first imaging control device C10 and the second imaging control device C20 may each include a trained model calculation unit that generates a first trained model and a second trained model based on parameters obtained from the first information processing device P10 and the second information processing device P20, and may be configured to determine whether the person included in the image information obtained by the first imaging control device C10 and the second imaging control device C20 is the same person as the first person and whether the person is the same person as the second person, based on the generated first trained model and the second trained model.

[0051] With this configuration, the information processing system 1 is configured to search for a person to be searched for using a trained model whose parameters have been adjusted to match the person to be searched for by the user of the first information processing device P10 and the user of the second information processing device P20, from image information acquired by the first information processing device P10, the second information processing device P20, the first image control device C10, and the second image control device C20.

[0052] Next, the hardware configuration of the first information processing device P10 will be described with reference to Figures 2 and 3. Figure 2 is a diagram showing an example of the hardware configuration of the first information processing device P10, and Figure 3 is a diagram showing an example of the hardware configuration of the first information processing device P10 that is different from Figure 2. For example, as shown in Figure 2, the first information processing device P10 is composed of a computer having a processor 10a, memory 10b, and I / O port 10c, and is configured so that the processor 10a reads and executes a program stored in the memory 10b.

[0053] Furthermore, as shown in Figure 3, for example, the first information processing device P10 is composed of a computer having a processing circuit 10d, which is dedicated hardware, and an I / O port 10c. The processing circuit 10d is composed of, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the first information processing device P10 is realized by these processors 10a or the processing circuit 10d, which is dedicated hardware, executing a program. Note that the first information processing device P10 may have hardware other than those described above. For example, the first information processing device P10 may include, not shown, an imaging device, a display device that displays information as images, a sound notification device that outputs information as sound, an input device that accepts user input operations, a communication device for making calls with external devices, etc. Furthermore, the hardware configuration of the second information processing device P20, the first imaging control device C10, and the second imaging control device C20 is the same as that of the first information processing device P10, so a detailed explanation is omitted.

[0054] Next, with reference to Figures 1, 4, and 5, the details of the processing performed by the first information processing device P10 will be described. Figure 4 is a flowchart showing an example of the processing performed by the first information processing device P10 according to Embodiment 1 for searching for a first person.

[0055] As shown in Figure 4, when the first information processing device P10 starts processing, it first acquires a trained model (step ST01a). For example, in this process, the first information processing device P10 acquires a trained model for classifying people included in image information based on feature quantities that indicate the relative positions between multiple parts of a person included in the image information, using the first trained model acquisition unit P13.

[0056] When the first information processing device P10 performs the processing in step ST01a, it determines whether or not it has acquired first image information including the first person (step ST01b). In this process, the first information processing device P10 determines whether or not it has acquired image information including the first person, who is the person being searched for by the user of the first information processing device P10. The first information processing device P10 may be configured to acquire image information including the first person by identifying the image of the first person included in the image information, based on an input signal from an input device (not shown) that accepts input from the user of the first information processing device P10, if the image information acquired by the first image information acquisition unit P11 includes an image other than the image of the first person.

[0057] In the processing of step ST01b, if the first image information including the first person has not been acquired (NO in step ST01b), the first information processing device P10 waits until the first image information including the first person is acquired.

[0058] In step ST01b, if first image information including the first person is acquired (YES in step ST01b), the first information processing device P10 generates a first trained model based on the acquired first image information (step ST02). In this process, the first information processing device P10 generates a first trained model with adjusted parameters based on the trained model acquired in step ST01a and the first image information including the first person acquired in step ST01b, so as to whether the person included in the image information is the same person as the first person.

[0059] When the first information processing device P10 performs the processing in step ST02, it outputs the first trained model to an external device (step ST03). In this process, the first information processing device P10 outputs the first trained model, which was generated in the processing of step ST02, to each device of the information processing system 1 as an external device, in order to perform the search for the first person.

[0060] When the first information processing device P10 performs the processing in step ST03, it determines whether or not it has acquired determination information indicating that a person included in other image information is the same person as the first person (step ST04). In this process, the first information processing device P10 determines whether or not it has acquired determination information indicating that a person included in other image information other than the first image information, acquired by any device of the information processing system 1 including the first information processing device P10, is the same person as the first person, from either the first determination unit P14 or an external device.

[0061] In the processing of step ST04, if determination information indicating that a person included in other image information is the same person as the first person has not been obtained (NO in step ST04), the first information processing device P10 waits until determination information indicating that a person included in other image information other than the first image information, obtained by any device of the information processing system 1 including the first information processing device P10, is the same person as the first person has been obtained.

[0062] In the process of step ST04, if determination information is obtained indicating that a person included in other image information is the same person as the first person (YES in step ST04), the first information processing device P10 outputs termination information regarding the first person (step ST05). In this process, the first information processing device P10 decides to terminate the search for the first person based on the fact that determination information has been obtained indicating that a person included in other image information other than the first image information, which has been obtained by any device of the information processing system 1 including the first information processing device P10, is the same person as the first person, and outputs termination information to each device of the information processing system 1 as an external device, indicating that the determination of whether or not a person included in image information is the same person as the first person has been terminated.

[0063] After completing step ST05, the first information processing device P10 terminates the process for searching for the first person.

[0064] Figure 5 is a flowchart illustrating an example of the process performed by the first information processing device P10 according to Embodiment 1 for searching for a second person. As shown in Figure 5, when the first information processing device P10 starts processing, it first determines whether or not it has acquired a second trained model (step ST11). In this process, the first information processing device P10 determines whether or not it has acquired a second trained model from an external device for classifying whether or not a person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person who is the target of the search by the user of the second information processing device P20.

[0065] If the second trained model has not been acquired during the processing of step ST11 (NO in step ST11), the first information processing device P10 waits until the second trained model is acquired.

[0066] In the process of step ST11, if a second trained model is obtained (YES in step ST11), the first information processing device P10 stores the second trained model in the first storage unit (step ST12). In this process, the first information processing device P10 has the second trained model obtained in the process of step ST11 stored in the first storage unit P17 by the first storage control unit P16.

[0067] When the first information processing device P10 performs the processing in step ST12, it determines whether or not it has acquired image information including a person (step ST13). In this process, the first information processing device P10 determines whether or not the first image information acquisition unit P11 has acquired image information for determining whether or not a second person is included by the second trained model stored in the first storage unit P17 in the processing of step ST12.

[0068] In the process of step ST13, if image information including a person is acquired (YES in step ST13), the first information processing device P10 determines whether the person included in the image information is the same person as the second person (step ST14). In this process, the first information processing device P10 reads the second trained model stored in the first storage unit P17, inputs the image information acquired in the process of step ST13 into the second trained model, and determines whether the person included in the image information is the same person as the second person based on the output of the second trained model.

[0069] In step ST14, if the person included in the image information is the same person as the second person (YES in step ST14), the first information processing device P10 outputs determination information to the external device indicating that the person included in the image information is the same person as the second person (step ST15). In this process, the first information processing device P10 notifies the external device of the determination result in the first information processing device P10 by outputting determination information to the external device that includes the determination result that the person included in the image information acquired in step ST13 is the same person as the second person.

[0070] In step ST13, if image information including a person was not acquired (NO of step ST13), in step ST14, if the person included in the image information was not the same person as the second person (NO of step ST14), and in step ST15, the first information processing device P10 determines whether or not it has acquired termination information regarding the second person (step ST16). In this process, the first information processing device P10 determines whether or not it has acquired termination information from the external device as a response to the determination information output to the external device in step ST15. For example, if determination information including a determination result that the person included in the image information is the same person as the second person is output to the second information processing device P20 in step ST15, the user of the second information processing device P20, based on the content of the determination information, performs an input operation to an input device (not shown) that is communicably connected to the second information processing device P20 to terminate the search when the location of the second person has been confirmed. As a result, termination information is output from the second information processing device P20 to the first information processing device P10.

[0071] If, during the processing in step ST16, termination information regarding the second person has not been obtained (NO in step ST16), the first information processing device P10 returns the process to step ST13.

[0072] In the process of step ST16, if termination information regarding the second person is obtained (YES in step ST16), the first information processing device P10 deletes the second trained model (step ST17). In this process, the first information processing device P10 deletes the second trained model stored in the first storage unit P17 based on the fact that the search for the second person has ended.

[0073] After completing step ST17, the first information processing device P10 terminates the process for searching for the second person.

[0074] Furthermore, the process performed by the second information processing device P20 to search for the second person is the same as the process performed by the first information processing device P10 to search for the first person, and the process performed by the second information processing device P20 to search for the first person is the same as the process performed by the first information processing device P10 to search for the second person, so the explanation is omitted.

[0075] As described above, the first information processing device P10 according to Embodiment 1 includes: a first trained model acquisition unit P13 that acquires a trained model for classifying people included in image information based on feature quantities indicating the relative positions between multiple parts of a person included in the image information; a first image information acquisition unit P11 that acquires image information including a person; a first trained model calculation unit P12 that performs calculations to adjust the parameters of the trained model so as to classify whether a person included in the image information is the same person as a first person, based on the trained model acquired by the first trained model acquisition unit P13 and the first image information including a specific first person acquired by the first image information acquisition unit P11; and a first output unit P15 that outputs the calculation results from the first trained model calculation unit P12 to an external device.

[0076] For example, the first information processing device P10 includes a first determination unit P14 that acquires determination information, including the determination result of whether the person included in the other image information is the same person as the first person, from a second information processing device P20, which is an external device that determines whether the person included in the other image information is the same person as the first person based on the calculation result of the first learned model calculation unit P12 and other image information. The first output unit P15 is configured to output the determination information acquired by the first determination unit P14 to a notification device for notifying the user.

[0077] With this configuration, even when the first information processing device P10 outputs the calculation results related to a trained model for classifying whether a person included in image information is the same person as the person being searched for, to an external device via an open communication network, it does not need to output information indicating the face image of the person being searched for, thus protecting the privacy of the person being searched for. For this reason, the first information processing device P20 according to Embodiment 1 makes it possible to search for a person using multiple external devices connected via the open communication network NT1 while protecting the privacy of the person being searched for.

[0078] Furthermore, the first information processing device P10 includes a first trained model acquisition unit P13 that acquires calculation results from an external device to adjust the parameters of a trained model that classifies whether a person included in image information is the same person as a specific person, based on feature quantities indicating the relative positions between multiple parts of a person included in image information, so that it classifies whether a person is the same person as a specific second person; a first storage control unit P16 that stores information indicating the calculation results acquired by the first trained model acquisition unit P13 in the first storage unit P17; and information acquired by the first trained model acquisition unit P13 and information acquired by the first image information acquisition unit P11. The system includes a first determination unit P14 that determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person, based on the image information obtained, and a first output unit P15 that outputs information indicating the determination result by the first determination unit P14 to an external device, and a first storage control unit P16 that deletes information regarding the calculation result acquired by the first trained model acquisition unit P13 stored in the first storage unit P17 based on the determination result by the first determination unit P14 which indicates that the person included in the image information acquired by the first image information acquisition unit P11 is the second person.

[0079] With this configuration, the first information processing device P10, for example, deletes information about the second person being searched for from the first storage unit P17 after the first information processing device P10 has finished searching for the second person, thereby protecting the privacy of the person being searched for.

[0080] Furthermore, the first information processing device P10 includes a first trained model acquisition unit P13 that acquires calculation results from an external device to adjust the parameters of a trained model that classifies whether a person included in image information is the same person as a specific person, based on feature quantities indicating the relative positions between multiple parts of a person included in image information, so that it classifies whether a person is the same person as a specific second person; a first storage control unit P16 that stores information indicating the calculation results acquired by the first trained model acquisition unit P13 in the first storage unit P17; and the first information processing device P10 that acquires The system includes a first determination unit P14 that determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the second person, based on the information obtained and the image information acquired by the first image information acquisition unit P11, and a first determination unit P14 that acquires termination information indicating that the determination by the first determination unit P14 has ended. The first storage control unit P16 is configured to delete the information regarding the calculation result acquired by the first trained model acquisition unit P13 stored in the first storage unit P17 when the termination information is acquired by the first determination unit P14.

[0081] With this configuration, the first information processing device P10, for example, deletes information about the second person being searched for from the first storage unit P17 after the first information processing device P10 has finished searching for the second person, thereby protecting the privacy of the person being searched for.

[0082] In Embodiment 1, the information processing system 1 is composed of two information processing devices, a first information processing device P10 and a second information processing device P20, and two imaging control devices, a first imaging control device C10 and a second imaging control device C20, but is not limited to this. The information processing system may be composed of only two or more information processing devices, or it may be composed of one information processing device and two or more imaging control devices.

[0083] Embodiment 2. Next, with reference to Figure 6, the information processing system 1A according to Embodiment 2 will be described. The information processing system 1A according to Embodiment 2 differs from the information processing system 1 according to Embodiment 1 in that it has a different number of devices, but the other configurations are the same, and the same names and reference numerals as in Embodiment 1 will be used for the same configurations as in Embodiment 1, and their descriptions will be omitted.

[0084] Figure 6 is a block diagram showing the schematic configuration of the information processing system 1A according to Embodiment 2. As shown in Figure 6, the information processing system 1A according to Embodiment 2 is configured such that three or more information processing devices, consisting of a first information processing device P10, a second information processing device P20, ..., and an nth information processing device Pn0, and three or more imaging control devices, consisting of a first imaging control device C10, a second imaging control device C20, ..., and an nth imaging control device Cn0, are connected to each other via a communication network NT1 so that they can communicate with one another.

[0085] The hardware configuration of the first information processing device P10, the second information processing device P20, ..., the nth information processing device Pn0, and the first imaging control device C10, the second imaging control device C20, ..., the nth imaging control device Cn0 according to Embodiment 2 is the same as that of the first information processing device P10 according to Embodiment 1, so a description will be omitted.

[0086] Furthermore, the processing performed by the first information processing device P10, the second information processing device P20, ..., and the nth information processing device Pn0 according to Embodiment 2 is the same as that performed by the first information processing device P10 according to Embodiment 1, so a description will be omitted.

[0087] Thus, the configuration of the first information processing device P10 and the first imaging control device C10 allows for the simultaneous searching of three or more individuals, who are the search targets of each of the three or more information processing devices, in an information processing system in which three or more devices are connected via a communication network NT1.

[0088] Embodiment 3. Next, the information processing system 1B according to Embodiment 3 will be described with reference to Figures 7 to 9. The information processing system 1B according to Embodiment 3 differs from the information processing system 1 according to Embodiment 1 in that it is equipped with a model generation device that generates trained models, and in that some of the configurations of the first information processing device and the second information processing device are different, but other configurations are the same, and the same names and reference numerals as in Embodiment 1 will be used and their descriptions will be omitted.

[0089] Figure 7 is a block diagram showing the schematic configuration of the information processing system 1B according to Embodiment 3. As shown in Figure 7, the information processing system 1B according to Embodiment 3 comprises a first information processing device P10B, a second information processing device P20B, a model generation device M10, a first imaging control device C10, and a second imaging control device C20, which are connected wirelessly or via a wired connection through a communication network NT1 so that they can communicate with each other.

[0090] The first information processing device P10B, as an information processing device, includes a first image information acquisition unit P11, a first trained model calculation unit P12, a first trained model acquisition unit P13, a first determination unit P14B, a first output unit P15B, a first storage control unit P16B, and a first storage unit P17.

[0091] The first determination unit P14B, acting as a determination unit, determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person, based on the information acquired by the first trained model acquisition unit P13 and the image information acquired by the first image information acquisition unit P11. For example, the first determination unit P14B determines whether the person included in the image information acquired by the first image information acquisition unit P11 is the same person as the specific person that the trained model is searching for, based on the trained model acquired by the first trained model acquisition unit P13 and the image information acquired by the first image information acquisition unit P11.

[0092] Specifically, the first determination unit P14B determines whether the person in the image information acquired by the first image information acquisition unit P11 is the same person as the first person and whether the person in the image information acquired by the first trained model acquisition unit P13 is the same person as the first person and whether the person in the image information is the same person as the second person, based on the third trained model whose parameters have been adjusted to classify whether the person in the image information acquired by the first trained model acquisition unit P13 is the same person as the first person and whether the person in the image information is the same person as the second person, respectively. In other words, the first determination unit P14B determines whether the person in the image information is the same person as the first person and whether the person in the image information is the same person as the second person, respectively, by inputting the image information acquired by the first image information acquisition unit P11 to the third trained model acquired by the first trained model acquisition unit P13.

[0093] Furthermore, the first determination unit P14B acquires information regarding the determination result from an external device that determines whether the person included in the image information is the same person as the first person. Also, the first determination unit P14B acquires information regarding the determination result from an external device that determines whether the person included in the image information is the same person as the second person.

[0094] Furthermore, the first determination unit P14B obtains termination information from the model generation device M10 indicating that the determination of whether or not the person included in the image information is the same person as the specific person being searched by the information processing system 1B has been completed by one or more devices in the information processing system 1B. The first determination unit P14B may be configured to obtain termination information from the model generation device M10 indicating that the determination of whether or not the person included in the image information is the same person as the specific person being searched by the information processing system 1B has been completed by one or more devices in the information processing system 1B, for each person being searched, or it may be configured to obtain termination information indicating that the determination of whether or not the person included in the image information is the same person as the specific person being searched by the information processing system 1B has been completed for all searched individuals. Furthermore, the first determination unit P14B constitutes the determination information acquisition unit in Embodiment 2.

[0095] The first output unit P15B, acting as an output unit, outputs various types of information to at least one of an external device connected to the first information processing device P10B via the communication network NT1 and a device (not shown) provided by the first information processing device P10B. For example, the first output unit P15B outputs the parameters of the first trained model generated by the first trained model calculation unit P12, as a result of calculations performed by the first trained model calculation unit P12, to the model generation device M10, which acts as an external device.

[0096] The first memory control unit P16B, acting as a memory control unit, controls the storage of information by the first memory unit P17. For example, if a learned model is already stored in the first memory unit P17, and a new learned model is acquired from an external device by the first learned model acquisition unit P13, the first memory control unit P16B will store the new learned model in the first memory unit P17 in place of the learned model already stored in the first memory unit P17. Specifically, if a first learned model is already stored in the first memory unit P17, and a third learned model is acquired from the model generation device M10 by the first learned model acquisition unit P13, the first memory control unit P16B will store the third learned model in the first memory unit P17 in place of the first learned model already stored in the first memory unit P17. In other words, when the first learned model is stored in the first storage unit P17, and the first learned model acquisition unit P13 acquires a third learned model from the model generation device M10 as a new learned model, the first storage control unit P16B replaces the first learned model stored in the first storage unit P17 with the third learned model.

[0097] Furthermore, for example, when the first memory control unit P16B receives termination information from the model generation device M10, it causes the first memory unit P17 to delete the trained model acquired from the model generation device M10 that is stored in the first memory unit P17. Specifically, when the first memory control unit P16B receives termination information from the model generation device M10, it causes the first memory unit P17 to delete the third trained model that is stored in the first memory unit P17. Note that when the first memory control unit P16B receives termination information from the model generation device M10, it may be configured to leave in the first memory unit P17 the trained model before the parameters were adjusted to suit the first and second people.

[0098] As shown in Figure 7, the second information processing device P20B, as an information processing device, includes a second image information acquisition unit P21, a second trained model calculation unit P22, a second trained model acquisition unit P23, a second determination unit P24B, a second output unit P25B, a second memory control unit P26B, and a second memory unit P27. The functions of the second determination unit P24B, the second output unit P25B, and the second memory control unit P26B provided by the second information processing device P20 are the same as those of the first determination unit P14B, the first output unit P15B, and the first memory control unit P16B provided by the first information processing device P10B, respectively, so their explanation is omitted.

[0099] The model generation device M10 comprises a parameter acquisition unit M11, a trained model calculation unit M12, and a termination determination unit M13. For example, the model generation device M10 is a device operated by a security company, an infrastructure operating company, a public institution, etc., which manages the search for a specific person using the information processing system 1B.

[0100] The parameter acquisition unit M11 acquires parameters of the trained model from the first information processing device P10B and the second information processing device P20B. For example, the parameter acquisition unit M11 acquires the parameters of the first trained model generated by the first information processing device P10B from the first information processing device P10B, and acquires the parameters of the second trained model generated by the second information processing device P20B from the second information processing device P20B.

[0101] The trained model calculation unit M12 performs calculations to generate a trained model based on the parameters acquired by the parameter acquisition unit M11. For example, the trained model calculation unit M12 has a trained model in advance whose parameters have not been adjusted to match the first and second people, and which classifies people included in image information based on feature quantities that indicate the relative positions between multiple parts of the person included in the image information. Based on this trained model and the parameters of the first trained model and the second trained model acquired by the parameter acquisition unit M11, the unit generates a third trained model by federated learning by the first information processing device P10B and the second information processing device P20B to classify whether the person included in the image information is the same person as the first person and whether it is the same person as the second person, based on feature quantities that indicate the relative positions between multiple parts of the person included in the image information.

[0102] Furthermore, for example, the trained model calculation unit M12 has a pre-trained model whose parameters have not been adjusted to match the first and second people, and which is used to classify people included in image information based on feature quantities that indicate the relative positions between multiple parts of a person included in the image information. Based on this pre-trained model and the parameters of the first pre-trained model acquired by the parameter acquisition unit M11, it generates a first pre-trained model. Furthermore, for example, the trained model calculation unit M12 has a pre-trained model whose parameters have not been adjusted to match the first and second people, and which is used to classify people included in image information based on feature quantities that indicate the relative positions between multiple parts of a person included in the image information. Based on this pre-trained model and the parameters of the second pre-trained model acquired by the parameter acquisition unit M11, it generates a second pre-trained model. The trained model calculation unit M12 outputs the generated pre-trained models to each device of the information processing system 1B.

[0103] The termination determination unit M13 generates termination information indicating that the determination of whether the person included in the image information is the same person as the specific person being searched for by the information processing system 1B has been completed by one or more devices in the information processing system 1B. For example, the termination determination unit M13 acquires determination information from each device in the information processing system 1B, and based on the acquisition of determination information including a determination result indicating that the person included in the image information is the first person, and determination information including a determination result indicating that the person included in the image information is the second person, from one of the devices in the information processing system 1B, it decides to terminate the search for these first and second people, and generates termination information indicating that the determination of whether the person included in the image information is the same person as the first person and whether it is the same person as the second person has been completed.

[0104] Furthermore, for example, the termination determination unit M13 acquires determination information from each device of the information processing system 1B, and based on the acquisition of determination information from any device of the information processing system 1B that includes a determination result indicating that the person included in the image information is the first person, it decides to terminate the search for the first person and generates termination information indicating that the determination of whether or not the person included in the image information is the same person as the first person has been terminated. Furthermore, for example, the termination determination unit M13 acquires determination information from each device of the information processing system 1B, and based on the acquisition of determination information from any device of the information processing system 1B that includes a determination result indicating that the person included in the image information is the second person, it decides to terminate the search for the second person and generates termination information indicating that the determination of whether or not the person included in the image information is the same person as the second person has been terminated.

[0105] Furthermore, the termination determination unit M13 may be configured to generate termination information based on the acquisition of determination information, as well as the acquisition of a signal indicating that the operator of the model generation device M10 has performed an input operation to terminate the search to an input device (not shown) that is communicably connected to the model generation device M10. For example, the operator of the model generation device M10 performs an input operation to terminate the search to the input device when the location of the person to be searched is confirmed based on the information regarding the location of the person to be searched included in the determination information. The termination determination unit M13 outputs the generated termination information to each device of the information processing system 1B.

[0106] Furthermore, the termination determination unit M13 may be configured to output termination information for each person being searched, indicating that the information processing system 1B has finished determining whether the person included in the image information is the same person as the specific person being searched by the information processing system 1B. Alternatively, it may be configured to output termination information indicating that the determination of whether the person included in the image information is the same person as the specific person being searched by the information processing system 1B has finished for all search targets. For example, if the termination determination unit M13 is configured to output termination information for each person being searched, the model generation device M10 may be configured to output a new trained model, whose parameters have been adjusted to search only for people whose search has not yet been completed, to each device of the information processing system 1B.

[0107] The hardware configurations of the first information processing device P10B, the second information processing device P20B, and the model generation device M10 are the same as those of the first information processing device P10 according to Embodiment 1, so their description will be omitted.

[0108] Next, with reference to Figures 7 to 9, the details of the processing performed by the first information processing device P10B and the model generation device M10 will be described. Figure 8 is a flowchart showing an example of the processing performed by the first information processing device P10B according to Embodiment 3 for searching for a first person and a second person. Note that some of the processing performed by the first information processing device P10B according to Embodiment 3 is the same as the processing performed by the first information processing device P10 according to Embodiment 1, and the processing that is the same as in Embodiment 1 will not be explained.

[0109] As shown in Figure 8, when the first information processing device P10B starts processing, it first acquires a trained model (step ST01a). After performing the processing in step ST01a, the first information processing device P10B determines whether or not it has acquired first image information including the first person (step ST01b). If, in the processing of step ST01b, the first image information including the first person has not been acquired (NO in step ST01b), the first information processing device P10B waits until the first image information including the first person is acquired. If, in the processing of step ST01b, the first image information including the first person has been acquired (YES in step ST01b), the first information processing device P10B generates a first trained model based on the acquired first image information (step ST02).

[0110] When the first information processing device P10B performs the processing in step ST02, it outputs the parameters of the first trained model to the model generation device M10 (step ST23). In this process, the first information processing device P10B outputs the parameters of the first trained model, which was generated in the processing of step ST02 based on the trained model acquired in the processing of step ST01a and the first image information including the first person acquired in the processing of step ST01b, to the model generation device M10.

[0111] When the first information processing device P10B performs the processing in step ST23, it determines whether or not it has obtained a new trained model from the model generation device M10 (step ST24). In this process, the first information processing device P10B determines whether or not it has obtained a new trained model from the model generation device M10 for searching for a specific person who is the target of the search. For example, in the processing of step ST23, the first information processing device P10B obtains from the model generation device M10 a new trained model that was generated by the model generation device M10 based on the parameters output from the first information processing device P10B, for classifying whether or not a person included in image information is the same person as the first person. Furthermore, for example, the first information processing device P10B obtains a third trained model from the model generation device M10, which is a new trained model for classifying whether a person included in the image information is the same person as the first person and whether a person is the same person as the second person. This model is generated by the model generation device M10 based on the parameters output from the first information processing device P10B and the parameters output from the second information processing device P20B in the processing of step ST23.

[0112] In the process of step ST24, if no new trained model is obtained from the model generation device M10 (NO in step ST24), the first information processing device P10B waits until a new trained model is obtained from the model generation device M10.

[0113] In step ST24, if a new trained model is obtained from the model generation device M10 (YES in step ST24), the first information processing device P10B stores the new trained model in place of the already stored trained model (step ST25). In this process, the first information processing device P10B deletes the trained models stored in the first storage unit P17 that have not had their parameters adjusted for the first and second people, as well as the first trained model, and stores the new trained model obtained from the model generation device M10 in step ST24 in the first storage unit P17.

[0114] When the first information processing device P10B performs the processing in step ST25, it determines whether or not it has acquired image information containing a person (step ST26). In this process, the first information processing device P10B acquires image information that is the target of determining whether or not it contains a specific person who is the target of the search, using the first image information acquisition unit P11.

[0115] In the process of step ST26, if image information including a person is acquired (YES in step ST26), the first information processing device P10B determines whether or not a trained model for searching for the first person is stored (step ST27). In this process, the first information processing device P10B determines whether or not the trained model stored in the first storage unit P17 is a trained model whose parameters have been adjusted for searching for the first person. For example, when the first information processing device P10B acquires a trained model from the model generation device M10, it acquires information for identifying the person to be searched along with the trained model.

[0116] In step ST27, if a trained model for searching for the first person is stored (YES in step ST27), the first information processing device P10B determines whether the person included in the image information is the same person as the first person (step ST28). In this process, the first information processing device P10B inputs the image information acquired in step ST26 into the trained model stored in the first storage unit P17, and determines whether the person included in the image information is the same person as the first person based on the output of the trained model.

[0117] In the process of step ST28, if the person included in the image information is the same person as the first person (YES in step ST28), the first information processing device P10B outputs determination information to the model generation device M10 indicating that the person included in the image information is the same person as the first person (step ST29). For example, in this process, the first information processing device P10B outputs determination information to the model generation device M10 that includes the determination result that the person included in the image information acquired in the process of step ST26 is the same person as the first person, thereby enabling the operator of the model generation device M10 to decide whether or not to confirm the location of the first person and to end the search for the first person based on the determination information.

[0118] In step ST27, if a trained model for searching for the first person is not stored (NO of step ST27), in step ST28, if the person included in the image information is not the same person as the first person (NO of step ST28), or if step ST29 is performed, the first information processing device P10B determines whether a trained model for searching for the second person is stored (step ST30). In this process, the first information processing device P10B determines whether the trained model stored in the first storage unit P17 is a trained model whose parameters have been adjusted for searching for the second person.

[0119] In the process of step ST30, if a trained model for searching for a second person is stored (YES in step ST30), the first information processing device P10B determines whether the person included in the image information is the same person as the second person (step ST31). In this process, the first information processing device P10B inputs the image information acquired in the process of step ST26 into the trained model stored in the first storage unit P17, and determines whether the person included in the image information is the same person as the second person based on the output of the trained model.

[0120] In the process of step ST31, if the person included in the image information is the same person as the second person (YES in step ST31), the first information processing device P10B outputs determination information to the model generation device M10 indicating that the person included in the image information is the same person as the second person (step ST32). In this process, for example, the first information processing device P10B outputs determination information to the model generation device M10 that includes the determination result that the person included in the image information acquired in the process of step ST26 is the same person as the second person, thereby enabling the operator of the model generation device M10 to decide whether or not to confirm the location of the second person and to end the search for the second person based on the determination information.

[0121] In step ST26, if image information including a person has not been acquired (NO of step ST26), in step ST30, if a trained model for searching for a second person has not been stored (NO of step ST30), in step ST31, if the person included in the image information is not the same person as the second person (NO of step ST31), and in step ST32, the first information processing device P10B determines whether or not it has acquired termination information (step ST33). In this process, the first information processing device P10B determines whether or not it has acquired termination information from the model generation device M10 indicating that the first information processing device P10B has terminated its determination of whether or not the person included in the image information is the same person as the specific person being searched for.

[0122] If termination information has not been obtained during the processing in step ST33 (NO in step ST33), the first information processing device P10B returns the process to step ST24.

[0123] In the process of step ST33, if termination information is obtained (YES in step ST33), the first information processing device P10B deletes the trained model (step ST34). In this process, the first information processing device P10B protects the privacy of the person being searched by deleting the trained model stored in the first storage unit P17 for searching for the first and second person, based on the fact that it has obtained termination information from the model generation device M10.

[0124] After performing the process in step ST34, the first information processing device P10B terminates the process for searching for the first and second persons.

[0125] The process performed by the second information processing device P20B for searching for the first and second individuals is the same as the process performed by the first information processing device P10B for searching for the first and second individuals, so its explanation is omitted.

[0126] Figure 9 is a flowchart illustrating an example of the processing performed by the model generation device M10 according to Embodiment 3. More specifically, Figure 9 is a flowchart illustrating an example of the processing performed by the model generation device M10 when the first information processing device P10B and the second information processing device P20B according to Embodiment 3 perform a search for the first person and the second person. As shown in Figure 9, when the model generation device M10 starts processing, it first determines whether it has acquired parameters from either the first trained model or the second trained model (step ST41). In this process, the model generation device M10 determines whether one or both of the parameters of the first trained model generated by the first information processing device P10B and the parameters of the second trained model generated by the second information processing device P20B have been acquired from the first information processing device P10B and the second information processing device P20B. For example, in this process, the model generation device M10 acquires the parameters output in step ST23 of the flowchart shown in Figure 8.

[0127] If, during the processing of step ST41, neither the parameters of the first trained model nor the second trained model have been acquired (NO in step ST41), the model generation device M10 waits until parameters of either the first trained model or the second trained model have been acquired.

[0128] In step ST41, if parameters of either the first trained model or the second trained model have been obtained (YES in step ST41), the model generation device M10 generates a trained model according to the parameters (step ST42). For example, in this process, if the parameters of both the first trained model and the second trained model have been obtained in step ST41, the model generation device M10 generates a third trained model whose parameters have been adjusted to suit the first and second people based on these parameters. Alternatively, if only the parameters of the first trained model have been obtained in step ST41, the model generation device M10 may generate a first trained model whose parameters have been adjusted to suit only the first person, or if only the parameters of the second trained model have been obtained in step ST41, the model generation device M10 may generate a second trained model whose parameters have been adjusted to suit only the second person.

[0129] After performing the processing in step ST42, the model generation device M10 outputs the trained model to an external device (step ST43). In this process, the model generation device M10 outputs the trained model generated in step ST42 to each device of the information processing system 1B, which is an external device.

[0130] When the model generation device M10 performs the processing in step ST43, it determines whether or not it has acquired determination information indicating that the person included in the image information is the same person as either the first person or the second person (step ST45). In this process, the model generation device M10 determines whether or not it has acquired determination information indicating that the determination that the person included in the image information acquired by any device of the information processing system 1B is the same person as either the first person or the second person was made based on the trained model output in the processing of step ST43.

[0131] In the process of step ST45, if determination information is obtained indicating that the person included in the image information is the same person as either the first person or the second person (YES in step ST45), the model generation device M10 decides whether or not to terminate the search (step ST46). In this process, the model generation device M10 decides whether or not to terminate the search for a specific person by the information processing system 1B based on the fact that determination information has been obtained indicating that the person included in the image information acquired by any device of the information processing system 1B is the same person as either the first person or the second person.

[0132] For example, if the model generation device M10 has already completed the search for the second person, and in step ST45, it obtains determination information indicating that the person included in the image information is the same person as the first person, it decides to terminate the search for the specific person by the information processing system 1B based on a signal from the operator who obtained the determination information to an input device (not shown) indicating that the search should be terminated. Alternatively, if the model generation device M10 has not yet completed the search for the second person, and in step ST45, it obtains determination information indicating that the person included in the image information is the same person as the first person, it decides not to terminate the search for the specific person by the information processing system 1B.

[0133] If, in the processing of step ST45, determination information indicating that the person included in the image information is the same person as either the first person or the second person is not obtained (NO of step ST45), or if, in the processing of step ST46, it is decided not to terminate the search (NO of step ST46), the model generation device M10 returns the process to step ST41.

[0134] In the process of step ST46, if it is decided to terminate the search (YES in step ST46), the model generation device M10 outputs termination information to an external device (step ST47). In this process, based on the decision in step ST46 to terminate the search for a specific person by the information processing system 1B, the model generation device M10 outputs termination information to each device of the information processing system 1B indicating that the search for a specific person by the information processing system 1B has ended.

[0135] The model generation device M10 terminates processing after performing the process in step ST47.

[0136] As described above, the information processing system 1B according to Embodiment 3 is configured to output a trained model, generated based on the parameters of the trained model acquired from each information processing device, to each device of the information processing system 1B, and for each of these devices to search for the person to be searched.

[0137] With this configuration, the information processing system 1B can suppress opportunities for the trained model itself to be sent and received between each device, thereby protecting the privacy of the person being searched.

[0138] In any of the embodiments described above, the information processing device may have some or all of the functions of devices other than the information processing device in the information processing system described above, some of the configurations of the information processing device may be provided in other devices that are communicatively connected to the information processing device, or specific functions of the information processing device may be configured to be performed by a plurality of devices that are formed independently of each other.

[0139] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment.

[0140] The information processing device relating to this disclosure can be used to search for a specific person based on image information by multiple devices connected to each other via a communication network.

[0141] 1 Information processing system, 1A Information processing system, 1B Information processing system, 10a Processor, 10b Memory, 10c I / O port, 10d Processing circuit, C10 First imaging control device, C11 Image information acquisition unit, C13 Learned model acquisition unit, C14 Judgment unit, C15 Output unit, C16 Memory control unit, C17 Memory unit, C20 Second imaging control device, Cn0 nth imaging control device, M10 Model generation device, M11 Parameter acquisition unit, M12 Learned model calculation unit, M13 Termination judgment unit, NT1 Communication network, P10 First information processing device (information processing device), P10B First information processing device (information processing device), P11 First image information acquisition unit (image information acquisition unit), P12 First learned model calculation unit (learned model calculation unit), P13 First learned model acquisition unit (learned model acquisition unit, calculation result acquisition unit), P14 P14B First determination unit (determination unit, determination information acquisition unit), P15 First output unit (output unit), P15B First output unit (output unit), P16 First memory control unit (memory control unit), P16B First memory control unit (memory control unit), P17 First memory unit (memory unit), P20 Second information processing unit (information processing unit), P20B Second information processing unit (information processing unit), P21 Second image information acquisition unit (image information acquisition unit), P22 Second trained model calculation unit (trained model calculation unit), P23 Second trained model acquisition unit (trained model acquisition unit, calculation result acquisition unit), P24 Second determination unit (determination unit, determination information acquisition unit), P24B Second determination unit (determination unit, determination information acquisition unit), P25 Second output unit (output unit), P25B Second output unit (output unit), P26 Second memory control unit (memory control unit), P26B Second memory control unit (memory control unit), P27 Second memory unit (memory unit), Pn0 nth information processing device (information processing device).

Claims

1. An information processing device comprising: a trained model acquisition unit that acquires a trained model for classifying people included in image information based on feature quantities indicating the relative positions between multiple parts of a person included in the image information; an image information acquisition unit that acquires image information including a person; a trained model calculation unit that performs calculations to adjust the parameters of the trained model so as to classify whether a person included in the image information is the same person as the first person, based on the trained model acquired by the trained model acquisition unit and first image information including a specific first person acquired by the image information acquisition unit; and an output unit that outputs the calculation results from the trained model calculation unit to an external device.

2. The information processing apparatus according to claim 1, comprising: a determination information acquisition unit that acquires determination information including the determination result of whether or not the person included in the other image information is the same person as the first person from an external device that determines whether or not the person included in the other image information is the same person as the first person based on the calculation result of the trained model calculation unit and other image information, and the output unit outputs the determination information acquired by the determination information acquisition unit to a notification device for notifying the user.

3. An information processing apparatus according to claim 1 or 2, comprising: an calculation result acquisition unit that acquires calculation results from an external device to adjust the parameters of a trained model that classifies whether a person included in image information is the same person as a specific person based on feature quantities indicating the relative positions between multiple parts of a person included in image information, so as to classify whether a person is the same person as a specific second person; a storage control unit that stores information indicating the calculation results acquired by the calculation result acquisition unit in a storage unit; and a determination unit that determines whether a person included in image information acquired by the image information acquisition unit is the same person as the second person, based on the information acquired by the calculation result acquisition unit and the image information acquired by the image information acquisition unit, wherein the output unit outputs information indicating the determination result by the determination unit to an external device, and the storage control unit deletes the information regarding the calculation results acquired by the calculation result acquisition unit stored in the storage unit based on the determination result by the determination unit indicating that the person included in image information acquired by the image information acquisition unit is the second person.

4. An information processing apparatus according to any one of claims 1 to 3, comprising: a calculation result acquisition unit that acquires calculation results for adjusting the parameters of a trained model that classifies whether a person included in image information is the same person as a specific person based on feature quantities indicating the relative positions between multiple parts of a person included in image information from an external device, so as to classify whether a person is the same person as a specific second person; a storage control unit that stores information indicating the calculation results acquired by the calculation result acquisition unit in a storage unit; a determination unit that determines whether a person included in image information acquired by the image information acquisition unit is the same person as the second person based on the information acquired by the calculation result acquisition unit and image information acquired by the image information acquisition unit; and a determination information acquisition unit that acquires termination information indicating that the determination by the determination unit has ended, wherein the storage control unit deletes the information regarding the calculation results acquired by the calculation result acquisition unit stored in the storage unit when the termination information is acquired by the determination information acquisition unit.

5. A program to cause a computer to function as: a trained model acquisition unit that acquires a trained model for classifying people included in image information based on feature quantities indicating the relative positions between multiple parts of a person included in the image information; an image information acquisition unit that acquires image information including people; a trained model calculation unit that performs calculations to adjust the parameters of the trained model so as to classify whether the person included in the image information is the same person as the first person, based on the trained model acquired by the trained model acquisition unit and first image information including a specific first person acquired by the image information acquisition unit; and an output unit that outputs the calculation results of the trained model calculation unit to an external device.

6. An information processing method performed by an apparatus comprising a trained model acquisition unit, an image information acquisition unit, a trained model calculation unit, and an output unit, the method comprising: the trained model acquisition unit acquiring a trained model for classifying a person included in image information based on feature quantities indicating the relative positions between multiple parts of a person included in the image information; the image information acquisition unit acquiring image information including a person; the trained model calculation unit performing calculations to adjust the parameters of the trained model to classify whether a person included in image information is the same person as the first person, based on the trained model acquired by the trained model acquisition unit and first image information including a specific first person acquired by the image information acquisition unit; and the output unit outputting the calculation results from the trained model calculation unit to an external device.

Images