Information processing device, information processing method, and program

Abstract

To perform imaging while maintaining an appropriate distance from an object to be imaged.SOLUTION: An information processing device acquires and records state information indicating the state of an object to be imaged. The movement of an unmanned mobile body that images the object is controlled in accordance with approach permissible distance information, which indicates a distance at which the object can be approached and is obtained based on the recorded state information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a technology for controlling the movement of an unmanned vehicle. [Background technology]

[0002] Conventionally, in controlling the movement of unmanned moving bodies for the purpose of photography, a method has been adopted to avoid collisions with the subject. In this regard, Patent Document 1 discloses a method for photography using a drone, in which the state of the object relative to the drone is estimated based on an image obtained by photographing the object, and flight operations are changed according to the estimated state. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-46058 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, the state of the object relative to the drone changes immediately after capturing an image, and even if the drone's flight behavior is changed in accordance with the changed state, there is a possibility that the drone will come into contact with the object. For example, when capturing an image using a drone, it is assumed that the object is moving away from the drone, and as the drone flies toward the object, the object suddenly changes direction and moves toward the drone. In this case, even if the state of the approaching object is estimated and the drone's direction of movement is changed, the drone continues to move by inertia for a while, which could result in contact with the object. [Means for solving the problem]

[0005] An information processing device according to one aspect of the present disclosure is characterized by having a status information acquisition means for acquiring status information indicating the status of an object to be photographed, a recording means for recording the acquired status information, a distance information acquisition means for acquiring approachable distance information indicating the distance at which the object can be approached based on the recorded status information, and a control means for controlling the movement of an unmanned mobile body photographing the object in accordance with the acquired approachable distance information. [Effects of the Invention]

[0006] According to the technology of the present disclosure, it is possible to photograph an object to be photographed while maintaining an appropriate distance from the object. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of an information processing system. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of each device included in the information processing system. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of each device included in the information processing system. [Figure 4] 10 is a flowchart showing the flow of pre-processing. [Figure 5] 10A and 10B are diagrams showing an example of a UI screen for registering photographing object data and an example of a record of the photographing object data. [Figure 6] FIG. 10 is a diagram showing an example of an individual data record. [Figure 7] 10A and 10B are diagrams showing an example of a UI screen for selecting individual data to be linked to a photographing subject and an example of a record of data linking a photographing subject and an individual. [Figure 8] 10 is a flowchart showing the flow of a movement control process for an unmanned moving body. [Figure 9] FIG. 10 is a diagram showing an example of a record of data for estimating the state of a subject to be photographed; [Figure 10] FIG. 10 is a diagram showing an example of a record of time-series data of the state of a subject to be photographed. [Figure 11] FIG. 10 is a diagram illustrating an example of a record of approachable distance data. [Figure 12] FIG. 10 is a diagram showing an example of a record of difference data between a photographed object and an individual. [Figure 13] FIG. 1 is a schematic diagram illustrating a configuration of an information processing system. [Figure 14] FIG. 2 is a diagram illustrating an example of a functional configuration of each device included in the information processing system. [Figure 15] 10A and 10B are diagrams showing an example of a UI screen for registering photographing object data and an example of a record of the photographing object data. [Figure 16] FIG. 10 is a diagram showing an example of an individual data record. [Figure 17] 10A and 10B are diagrams showing an example of a UI screen for selecting individual data to be linked to a photographing subject and an example of a record of data linking a photographing subject and an individual. [Figure 18] 10 is a flowchart showing the flow of a movement control process for an unmanned moving body. [Figure 19] FIG. 10 is a diagram illustrating an example of a record of data for estimating a state of a photographing target. [Figure 20] FIG. 10 is a diagram showing an example of a record of data for estimating the surrounding state of a subject to be photographed; [Figure 21] FIG. 10 is a diagram showing an example of a record of time-series data of the state of a subject to be photographed. [Figure 22] FIG. 10 is a diagram illustrating an example of a record of approachable distance data. [Figure 23] FIG. 10 is a diagram showing an example of a record of difference data between a photographed object and an individual. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments for implementing the technology of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the technology of the present disclosure according to the claims. Not all combinations of features described in the embodiments are necessarily essential as solutions for the technology of the present disclosure, and multiple features may be combined arbitrarily. Note that identical configurations will be described with the same reference numerals. Also, each process (step) in a flowchart will be denoted with an "S" at the beginning.

[0009] <<Embodiment 1>> In this embodiment, a mode will be described in which an information processing system is used to acquire approachable distance information indicating the distance at which an unmanned moving body can approach a pet, with the pet being the subject of the image capture.

[0010] 1 is a schematic diagram showing an example of the configuration of an information processing system according to this embodiment. The information processing system according to this embodiment is a system for acquiring approachable distances, and includes a server device 111, an unmanned mobile object 112, and a terminal device 113. These devices 111 to 113 are connected via a network 114 so as to be able to send and receive data.

[0011] The server device 111 acquires the calculated approachable distance. The approachable distance is the distance between the photographing target 122 and the unmanned moving body 112, and indicates the distance at which the photographing target 122 can be safely photographed without being attacked by the unmanned moving body 112, such as by jumping on it, making it impossible to photograph the photographing target 122.

[0012] The unmanned mobile body 112 is a drone or the like that flies freely. Note that the unmanned mobile body 112 will be described as one that moves by flying like a drone, but the unmanned mobile body 112 is not limited to one that moves by flying, and may be one that moves by running on the floor or the like, for example, like an autonomous robot. The terminal device 113 is an information processing device such as a user terminal used by the user 121. The network 114 includes a wireless LAN and a wired LAN. The network 114 also includes the Internet.

[0013] A user 121 uses the information processing system of this embodiment by holding a terminal device 113, and issues instructions to the information processing system of this embodiment using the terminal device 113. A subject 122 to be photographed is a pet or the like.

[0014] (Hardware configuration of each device) 2 is a diagram showing an example of the hardware configuration of each device included in the information processing system of this embodiment. The server device 111 includes a CPU 211, a ROM 212, a RAM 213, a communication unit 214, and a storage medium 215, and each device is connected to a bus 216 so that data can be transmitted and received between them.

[0015] The CPU 211 is a control unit that is made up of at least one processor or circuit, and controls the entire server device 111 .

[0016] The ROM 212 is an electrically erasable and recordable memory that stores constants, programs, etc. for the operation of the CPU 211. The programs store various processing programs and various data, which will be described later. The RAM 213 loads constants, variables, programs, etc. for the operation of the CPU 211 read from the ROM 212.

[0017] The communication unit 214 is an interface for communicating with external devices such as network devices or USB devices, and performs data communication via a network or sending and receiving data to and from external devices. The storage medium 215 is a recording medium such as a memory card, and is composed of a semiconductor memory or the like.

[0018] The unmanned vehicle 112 has a CPU 221, a ROM 222, a RAM 223, a communication unit 224, a storage medium 225, an imaging unit 226, and a movement control unit 227, and each device is connected to a bus 228 so that data can be transmitted and received between them.

[0019] The CPU 221 is a control unit that is made up of at least one processor or circuit, and controls the unmanned vehicle 112 as a whole.

[0020] The ROM 222 is an electrically erasable and recordable memory, and stores constants, programs, etc. for the operation of the CPU 221. The programs are computer programs for executing various flowcharts described later in this embodiment.

[0021] In the RAM 223, constants and variables for the operation of the CPU 221, programs read from the ROM 222, etc. are developed.

[0022] The communication unit 224 is an interface for communicating with external devices such as network devices or USB devices, and performs data communication via a network or sending and receiving data to and from external devices. The storage medium 225 is a recording medium such as a memory card, and is composed of a semiconductor memory or the like.

[0023] The imaging unit 226 is an imaging element that converts an optical image into an electrical signal and is configured with a CCD, CMOS element, etc. The imaging unit 226 also has a plurality of microlenses on the imaging surface, and is capable of measuring distances based on phase difference.

[0024] The movement control unit 227 controls the movement of the unmanned mobile body, such as rotating blades such as propellers, and motors for rotating the blades. For example, the unmanned mobile body 112 has multiple rotating blades, and can move, such as flying along any route, by operation of the movement control unit 227.

[0025] The terminal device 113 has a CPU 231, a ROM 232, a RAM 233, a communication unit 234, a storage medium 235, and a display / input unit 236, and each device is connected to a bus 237 so that data can be transmitted and received between them.

[0026] The CPU 231 is a control unit that is made up of at least one processor or circuit, and controls the entire terminal device 113 .

[0027] The ROM 232 is an electrically erasable and recordable memory that stores constants, programs, etc. for the operation of the CPU 231. The programs are computer programs for executing various flowcharts described later in this embodiment. The RAM 233 stores constants, variables, programs, etc. for the operation of the CPU 231 read from the ROM 232.

[0028] The communication unit 234 is an interface for communicating with an external device such as a network device or a USB device, and performs data communication via a network or sending and receiving data to and from the external device.

[0029] The storage medium 235 is a recording medium such as a memory card, and is configured from a semiconductor memory or the like.

[0030] The display / input unit 236 is made up of buttons, a touch panel, etc., and a display device such as a liquid crystal monitor, and receives operation inputs from the user and displays the results of the operation.

[0031] (Functional configuration of each device) 3 is a block diagram showing an example of the functional configuration of each of the server device 111, unmanned mobile object 112, and terminal device 113 included in the information processing system of this embodiment. Details of each process and data will be explained later using flowcharts.

[0032] The server device 111 includes, as processing units, a shooting target estimation unit 301, a state recording unit 304, an approachable distance calculation unit 305, a control instruction unit 306, a shooting target recording unit 307, and a communication unit 214. The shooting target estimation unit 301 includes a recognition unit 302 and a state estimation unit 303. The server device 111 also stores, as data, learning data 308, time-series data of the state of the shooting target (hereinafter referred to as time-series data) 309, and information data 310 of the shooting target. Note that each unit that the server device 111 has as a functional configuration is realized by the CPU 211 executing a program stored in the ROM 212. The data 308 to 310 is stored in, for example, a storage medium 225.

[0033] The unmanned mobile object 112 includes a communication unit 224, an imaging unit 226, and a movement control unit 227. The terminal device 113 includes a communication unit 234 and a display / input unit 236.

[0034] Next, each data will be described. Learning data 308 is composed of individual data 317, approachable distance data 318, and state estimation data 319. Information data 310 of the photographed object is composed of photographed object data 320, linking data 321 between the photographed object and the individual, and difference data 322 between the photographed object and the individual. The server device 111, the unmanned vehicle 112, and the terminal device 113 each have their communication units 214, 224, and 234 connected to the network 114, thereby transmitting and receiving data.

[0035] The functional configuration of the server device 111 will be described. The photographed object estimation unit 301 estimates information about the photographed object. The photographed object estimation unit 301 has a recognition unit 302 and a state estimation unit 303. The recognition unit 302 performs detection using photographed object data 320 stored in photographed object information data 310 for images photographed and acquired by the imaging unit 226 of the unmanned mobile object 112, and recognizes the photographed object. That is, the recognition unit 302 recognizes the photographed object using individual data 317 stored in the training data 308. The state estimation unit 303 estimates the state of the photographed object using state estimation data 319 stored in the training data 308 for images photographed and acquired by the imaging unit 226 of the unmanned mobile object 112. That is, it can be said that the state estimation unit 303 acquires state information indicating the state (first state and second state) of the object of the photographed object.

[0036] The state recording unit 304 records the subject recognized by the recognition unit 302 and the state of the subject estimated by the state estimation unit 303 in time-series data 309 .

[0037] The approachable distance calculation unit 305 calculates the approachable distance to the object using the time-series data 309 and the approachable distance data 318 of the training data 308. That is, the approachable distance calculation unit 305 can be said to acquire approachable distance information indicating the approachable distance to the object. For example, the approachable distance calculation unit 305 may acquire the approachable distance information by inputting recorded state information, which is time-series data, into a trained model that has been trained to input the state of the object and output approachable distance information. The trained model may be trained to input the first and second states of the object and the elapsed time from when the object changes from the first state to the second state, and output the approachable distance information. The trained model is assumed to be pre-stored in, for example, the ROM 212 or the storage medium 215.

[0038] Based on the approachable distance to the subject calculated by the approachable distance calculation unit 305, the control instruction unit 306 issues an instruction to the movement control unit 227 of the unmanned mobile body 112 to approach the subject.

[0039] The photographing object recording unit 307 records information data 310 of the photographing object based on information inputted through the display / input unit 236 of the terminal device 113 .

[0040] Each functional unit included in the server device 111, the unmanned vehicle 112, and the terminal device 113 may also take on part of the functions of other functional units.

[0041] (Pre-processing) FIG. 4 is a flowchart showing the flow of the pre-processing. In S401, information about the subject to be photographed is registered and the registered subject to be photographed is saved. Specifically, the user 121 inputs information about the subject to be photographed using the UI screen displayed by the display / input unit 236 of the terminal device 113.

[0042] (UI screen for input) 5(a) is a diagram showing an example of an input UI screen. The input UI screen 510 is a screen for registering data about the pet to be photographed. The UI screen 510 has a pet name input field 511, a breed input field 512, an age input field 513, a personality input field 514, a body length input field 515, a weight input field 516, an attached photo path information input field 517, and an attached video path information input field 518. The UI screen 510 has an attach photo button 521 and an attach video button 522.

[0043] Each of the input fields 511 to 516 is an area that accepts data input for the corresponding item by user operation. Specifically, the pet name input field 511 allows the name of the pet to be photographed to be input, the age input field 513 allows the age of the pet to be photographed to be input, and the body length input field 515 allows the body length of the pet to be photographed to be input.

[0044] The breed input field 512 allows the breed of the pet to be photographed to be input. For example, if the pet to be photographed is a cat, breeds such as Persian, American Shorthair, Russian Blue, Japanese cat, mixed breed cat, etc. can be input. The personality input field 514 allows the personality of the pet to be photographed to be input. For example, names indicating personality types such as quiet, active, lonely, etc. can be input. Note that the input fields 512 to 516 are not limited to a format for direct input, and may also be a format for inputting information selected from a pull-down menu.

[0045] The photo attachment button 521 allows a photo of the pet to be photographed to be attached. When a photo of the pet is attached using the photo attachment button 521, password information is automatically entered in the password information input field 517 for the attached photo. The video attachment button 522 allows a video showing the normal behavior of the pet to be photographed, for example, a 1-minute or 5-minute video at 30 fps that captures the individual from all angles, to be attached. When a video of the pet is attached using the video attachment button 522, password information is automatically entered in the password information input field 518 for the attached video.

[0046] Although the UI screen 510 has been described as an example in which information for all items corresponding to the input fields 511 to 518 is input, the present invention is not limited to this. A user may simply input the name of the pet being photographed and attach a video of the pet being photographed. In this case, the photographed subject recording unit 307 estimates the breed, age, personality, body length, and weight of the pet being photographed based on the attached video and individual data 317, and inputs the estimated information into the corresponding input fields.

[0047] The information entered on the UI screen 510 is sent to the photographing object recording unit 307 via the communication units 234 and 214. The photographing object recording unit 307 converts the sent information into a format as shown in FIG. 5(b) and records it in the photographing object data 320 of the photographing object information data 310.

[0048] 5(b) holds information corresponding to each of the items: pet ID 531, pet name 532, breed 533, age 534, personality 535, body length 536, weight 537, image 538, and video 539. Note that the "1" in pet ID 531 is a number assigned by the photographic subject recording unit 307, but the items other than pet ID 531 hold information sent from the terminal device 113.

[0049] In S402, the individual ID of the learning data is linked to the pet. Specifically, the photographing object recording unit 307 of the server device 111 searches for records of individual data 317 in the learning data 308 that are similar to the photographing object data 320 recorded in S401.

[0050] (Individual data record) 6 is a diagram showing an example record of the individual data 317 of the training data. The individual data record 600 holds data on each individual, including an individual ID 601, breed 602, age 603, personality 604, body length 605, weight 606, and video 607.

[0051] The individual ID 601 indicates an ID for identifying an individual in the learning data. The breed 602, age 603, personality 604, body length 605, weight 606, and video 607 are the same as the corresponding items in the photographed subject data 320. When searching for records, each item other than the video is compared, differences are weighted and added, and records with differences below a certain level are determined to be similar records. In this embodiment, differences in personality and breed are weighted heavily (if there are differences, they are not included in the similar records), and body length, age, and weight are weighted lightly (even if there are some differences, they are included in the similar records). Note that the magnitude of the weighting is not limited to this, and weighting may be performed at any magnitude.

[0052] The photographing object recording unit 307 of the server device 111 sends records similar to the photographing object data 320 retrieved above to the terminal device 113 via the communication units 214 and 234. The sent data is displayed on the display / input unit 236 of the terminal device 113.

[0053] (UI screen for displaying similar records) 7A shows an example of a UI screen 700 for displaying similar records. The UI screen 700 is displayed on the display / input unit 236 of the terminal device 113.

[0054] The UI screen 700 is a screen for displaying similar records, and displays a list 710 of similar records. The list 710 displays information on each of the following items: individual ID 711, breed 712, age 713, personality 714, body length 715, weight 716, and video 717. The video 717 item further displays buttons 721-723 that accept user operations to view videos of each individual. The UI screen 700 also has an input field 724 for inputting the individual ID of the closest individual. When one of the buttons 721-723 is pressed by a user operation, a video showing the normal behavior of the individual corresponding to the pressed button is played. The input field 724 is an area that accepts user operation to input the individual that is closest to the pet based on the displayed records and videos.

[0055] When input into the display / input unit 236 of the terminal device 113 is completed, the user's input results are sent to the photographic object recording unit 307 of the server device 111 via the respective communication units 234, 214. The photographic object recording unit 307 records the linking data between the photographic object and the individual in the photographic object and individual linking data 321 in the photographic object information data 310.

[0056] (Data linking the photographed subject to the individual) 7(b) is a diagram showing an example of data linking the pet and the individual subject of photography. The pet and individual linking data 730 is stored by linking data for each item of pet ID 731 and individual ID 732. That is, the pet and individual linking data 730 is stored as data linking the pet ID 731, "1," and the individual ID 732, "5."

[0057] In this embodiment, we have described a case where a pet and an individual are linked by a user's input operation, but the system may also automatically estimate the record of the individual that is most similar to the pet from each video of the pet and the individual, and link the pet ID and individual ID based on the estimation result.

[0058] (Processing for deriving the approachable distance) Figure 8 is a flowchart showing the overall flow of processing executed by the information processing system. Note that in Figure 8, processing for deriving the approachable distance is executed. The unmanned mobile body 112 is in an automatic flying state, recognizing a target to be photographed based on images obtained by photographing the surroundings of the unmanned mobile body 112, and photographing the target by approaching or moving away from the recognized target or circling around the target.

[0059] In S801, the imaging target estimation unit 301 of the server device 111 estimates the state of the imaging target. That is, the captured images captured by the imaging unit 226 of the unmanned mobile body 112 are acquired via the respective communication units 224 and 214, and the recognition unit 302 and the state estimation unit 303 each perform processing on the acquired captured images.

[0060] Specifically, the recognition unit 302 determines whether or not a pet of the item is present in the captured image captured by the imaging unit 226, using an image stored in the image column of the record of the photographed object data 320 shown in Fig. 6. The recognition unit 302 also determines in which location (area) of the image the pet of the photographed object is present. The state estimation unit 303 estimates the distance between the photographed object 122 and the unmanned moving object 112 from the camera parameters of the imaging unit 226, the size of the photographed object in the captured image, etc. The estimated distance between the photographed object 122 and the unmanned moving object 112 is used in S804, which will be described in detail later.

[0061] The state estimation unit 303 estimates the state of the subject to be photographed based on the image captured by the imaging unit 226, the presence or absence of a pet recognized by the recognition unit 302, the location of the pet, and state estimation data 319 in the learning data 308.

[0062] (State estimation data record) FIG. 9 is a diagram showing an example of a record of state estimation data. A record 900 of the state estimation data is data held in the state estimation data 319. In the record 900 of the state estimation data, a state 901 is linked to learning data 902 for state estimation. The state 901 holds states indicating behaviors, emotions, etc., such as "sleeping state," "eating state," "normal state," "threatening state," and "running around state." The learning data 902 for state estimation holds images for each state, such as "image of sleeping state," "image of eating state," "image of normal state," "image of threatening state," and "image of running around state."

[0063] In the state estimation data record 900, "sleeping state" and "image of sleeping state," "eating state" and "image of eating state," "normal state" and "image of normal state," and "threatening state" and "image of threatening state" are linked together. Also, "running around state" and "image of running around state" are linked together. By using the state estimation data record 900 shown in FIG. 9, it can be estimated which state the photographed subject is in among "sleeping state," "eating state," "normal state," "threatening state," and "running around state."

[0064] In S802, the state recording unit 304 records the state of the subject estimated in S801 in the time-series data 309.

[0065] (Time series data records) 10 is a diagram showing an example of a record of time-series data of the state of a subject to be photographed. A record 1000 of the time-series data of the state of a subject to be photographed is held in the time-series data 309. The record 1000 of the time-series data of the state of a subject to be photographed holds information indicating time 1001, pet ID 1002, and state 1003 in chronological order, linked to each other.

[0066] Time 1001 indicates the time when the process of S801 was executed. Pet ID 1002 indicates an ID for identifying the pet that is the subject of photography recognized by the recognition unit 302. State 1003 indicates the state of the subject of photography estimated by the state estimation unit 303.

[0067] In the record 1000 of the time-series data of the state of the photographed object, only when the state changes are recorded, but it is also possible to record when the state does not change.

[0068] In S803, the approachable distance calculation unit 305 executes a process of calculating the approachable distance. The approachable distance calculation process S803 is performed using the approachable distance data record, which is the approachable distance data 318 of the learning data 308.

[0069] (Approachable distance data record) 11 is a diagram showing an example of a record of approachable distance data, which shows data in which approachable distances corresponding to state changes of individual ID=5 are recorded.

[0070] The approachable distance data record 1100 holds information on each of the items No. 1101, individual ID 1102, state before change 1103, elapsed time since state before change 1104, state after change 1105, and approachable distance 1106 in association with each other.

[0071] No. 1101 indicates an ID for identifying a record. Individual ID 1102 indicates an ID for identifying an individual. State before change 1103 indicates what state the state was before the state after the change. If the state before the change is still continuing, it will have the same value as the state after the change. If the state before the change and the state after the change are different, they will have different values. Elapsed time 1104 of the state before the change indicates the time that has passed since the state before the change. State after change 1105 indicates what state the state after the change is. Approachable distance 1106 indicates the distance that the unmanned mobile object was able to approach when it changed from the state before the change (first state) to the state after the change (second state) (the distance at which the pet started to move when the unmanned mobile object gradually approached the pet).

[0072] The states will be explained using record No. 3 as an example. The state before the change 1103 is "threatening state," the time elapsed since the state before the change 1104 is "2 minutes," and the state after the change 1105 is "normal state," which indicates that the normal state was reached after 2 minutes had passed since the threatening state. In other words, this indicates that the individual ID "5" is linked to the "threatening state" (first state), the "normal state" (second state), the time it took to change from the "threatening state" to the "normal state" of "2 minutes," and the approachable distance of "100 cm."

[0073] The approachable distance calculation process S803 is performed by finding the record of the approachable distance data in FIG. 11 that corresponds to the time-series data 1000 in FIG. 10. As an example, the calculation process of the approachable distance corresponding to the time "2023 / 2 / 3 13:02" in FIG. 10 will be described. From the time-series record 1000 in FIG. 10, it can be seen that the pet was in a threatening state at "2023 / 2 / 3 13:00" and changed to a normal state at "2023 / 2 / 3 13:02". Furthermore, regarding the link between the pet ID and the individual ID, it can be seen from the record example in FIG. 7(b) that the pet ID "1" is linked to the individual ID "5". From the above, it is matched with the record No. 3 in FIG. 11. As a result, the approachable distance is calculated to be 100 cm.

[0074] The above describes a case where information exists for a time identical to a certain time listed in the time-series data 1000 of FIG. 10 . However, this is not limited to this case, and the approachable distance calculation process may be performed at any timing. For example, assume that the approachable distance calculation process is performed at time "2023 / 2 / 3 13:01." Since no state associated with this time is recorded, this indicates that the state has remained unchanged since times prior to time "2023 / 2 / 3 13:01." A threatening state is recorded at time "2023 / 2 / 3 13:00," which is prior to time "2023 / 2 / 3 13:01." Based on these records, it is determined that the state has not changed for one minute, so this matches record No. 2 in the approachable distance data record 1100 of FIG. 11 . The approachable distance calculated from record No. 2 is 130 cm.

[0075] In S804, the control instruction unit 306 compares the approachable distance calculated in S803 with the distance between the subject 122 to be photographed and the unmanned mobile body 112 estimated in S801, and issues a control instruction to the movement control unit 227 of the unmanned mobile body 112 via the communication units 214 and 224.

[0076] The contents of the control instruction will be described. The control instruction unit 306 issues a control instruction to control the movement of the unmanned mobile body so that the distance between the subject 122 and the unmanned mobile body 112 becomes the same as the approachable distance. Specifically, if the approachable distance is greater than the distance between the subject 122 and the unmanned mobile body 112 estimated in S801, a movement instruction is issued to control the movement of the unmanned mobile body so that the distance between the subject 122 and the unmanned mobile body 112 becomes greater to the approachable distance. If the approachable distance is the same as the distance between the subject 122 and the unmanned mobile body 112 estimated in S801, a control instruction is issued to control the movement of the unmanned mobile body so that the distance between the subject 122 and the unmanned mobile body 112 remains the approachable distance. If the approachable distance is shorter than the distance between the subject 122 and the unmanned mobile body 112 estimated in S801, a control instruction is issued to control the movement of the unmanned mobile body so that the distance between the subject 122 and the unmanned mobile body 112 becomes smaller to the approachable distance.

[0077] The control method for the unmanned mobile body 112 is not limited to the flow shown in FIG. 8 , and feedback of the control results may be performed. For example, when the unmanned mobile body 112 approaches the photographic target 122 and the photographic target 122 starts moving toward the unmanned mobile body 112, information indicating the distance at which the photographic target 122 started to move is sent to the photographic target recording unit 307 via the communication units 224 and 214. This information indicating the distance is obtained in S801 by repeatedly executing the process shown in FIG. 8. The photographic target recording unit 307 may store information regarding each of the items shown in FIG. 12 , namely, the pet ID 1201, the state before change 1202, the elapsed time before change 1203, the state after change 1204, and the approachable distance 1205, as difference data 322 between the photographic target and the individual. Furthermore, when calculating the approachable distance, if data corresponding to the time-series data exists in the difference data 322 between the photographic target and the individual, the data corresponding to the time-series data is used to calculate the approachable distance. Furthermore, the movement control unit 227 controls movement to maintain a distance from the photographing target 122. That is, when the data corresponding to the time-series data is difference data 1200 between the photographing target and the individual, as shown in Fig. 12, the calculated approachable distance is 120 cm. Then, the movement of the unmanned moving body is controlled in accordance with the approachable distance being 120 cm.

[0078] That is, when the state of the subject to be photographed is different from the state of the subject to be photographed indicated by the status information used to acquire the approachable distance information at a distance different from the approachable distance indicated by the approachable distance information, the following data may be recorded as difference data 322 between the subject to be photographed and the individual. That is, data linking the state of the subject to be photographed indicated by the status information with the distance between the unmanned moving body and the subject to be photographed may be recorded as difference data 322 between the subject to be photographed and the individual. Then, difference data 322 between the subject to be photographed and the individual may be used when calculating the approachable distance.

[0079] As described above, according to this embodiment, it is possible to photograph an object to be photographed while maintaining an appropriate distance from the object.

[0080] <<Embodiment 2>> In this embodiment, a case where the subject of photography is a child will be described. In this embodiment, the differences from the first embodiment will be mainly described.

[0081] This embodiment is the same as embodiment 1, except that the screen of the display / input unit 236, the learning data 308, the time-series data 309, and the information data 310 of the subject to be photographed are changed from those for pets to those for children. Suppose there is a ball near a child.

[0082] 13 is a schematic diagram showing an example of the configuration of an information processing system according to this embodiment. The information processing system according to this embodiment is a system that acquires approachable distance information indicating the distance at which it is possible to approach a child who is the subject of photography 123. Also, suppose that there is a ball 124 that the child can throw within a specific area 1331 near the child who is the subject of photography 123. The specific area 1331 is the periphery of the subject of photography that indicates the range within which the child's hand or the like can reach, and is, for example, an area with a radius of about 30 cm to 50 cm centered on the child.

[0083] The information processing system of this embodiment has a server device 1311, an unmanned mobile body 112, and a terminal device 113, which are connected to each other so as to be able to send and receive data via a network 114. In Fig. 1, the subject 122 to be photographed is a pet, but in Fig. 15, the subject 123 to be photographed is a child, and the system is the same as in the first embodiment except that there is a ball 124 in a specific area (surroundings) 1331.

[0084] The hardware configuration of each device included in the information processing system of this embodiment is the same as the hardware configuration of each device included in the information processing system of embodiment 1 shown in Fig. 2, and detailed description thereof will be omitted. That is, the server device 1311 included in the information processing system of this embodiment has the same hardware configuration as the server device 111 included in the information processing system of embodiment 1.

[0085] (Functional configuration of each device) 14 is a block diagram showing an example of the functional configuration of each of the server device 1311, unmanned vehicle 112, and terminal device 113 included in the information processing system of this embodiment. Details of each process and data will be explained in the flowcharts described later. Note that the same functional units will be assigned the same reference numerals and detailed explanations thereof will be omitted.

[0086] The server device 1311 includes, as processing units, a shooting target estimation unit 1401, a state recording unit 1404, an approachable distance calculation unit 1405, a control instruction unit 306, a shooting target recording unit 1407, and a communication unit 214. The shooting target estimation unit 1401 includes a recognition unit 302 and a state estimation unit 1403. The server device 111 also stores, as data, learning data 1408, time-series data of the state of the shooting target (hereinafter referred to as time-series data) 1409, and information data 1410 of the shooting target. Note that each unit included in the functional configuration of the server device 1311 is realized by the CPU 211 executing a program stored in the ROM 212. The data 1408 to 1410 is stored in, for example, a storage medium 225.

[0087] In addition to estimating the state of the object to be photographed, the state estimation unit 1403 recognizes the surroundings of the object to be photographed by detecting the surroundings data 1433 stored in the information data 310 of the object to be photographed for the image captured by the imaging unit 226 of the unmanned mobile body 112. In other words, it can be said that the state estimation unit 1403 acquires state information indicating the state of the object to be photographed (first state and second state) and the surrounding state of the object.

[0088] The approachable distance calculation unit 1405 calculates the approachable distance to the object to be photographed using the time-series data 1409 and the approachable distance data 1418 of the training data 1408. In other words, it can be said that the approachable distance calculation unit 1405 acquires approachable distance information indicating the approachable distance to the object. For example, the approachable distance calculation unit 1405 may acquire the approachable distance information by inputting the recorded state information into a trained model that has been trained to receive the state of the object and the state of the object's surroundings as input and output the approachable distance information. The trained model may be trained to receive the first and second states of the object, the elapsed time until the object changes from the first state to the second state, and the state of the object's surroundings as input and output the approachable distance information. Note that the trained model is assumed to be stored in advance in, for example, the ROM 212 or the storage medium 215.

[0089] Each functional unit included in the server device 1311, the unmanned vehicle 112, and the terminal device 113 may also take on part of the functions of other functional units.

[0090] The pre-processing in this embodiment is the same as the pre-processing in the first embodiment shown in FIG. 4, and a detailed description thereof will be omitted.

[0091] In the pre-processing of this embodiment, information about the subject to be photographed is registered and the registered subject to be photographed is saved. Specifically, the user 121 inputs information about the subject to be photographed using the UI screen displayed by the display / input unit 236 of the terminal device 113.

[0092] (UI screen for input) 15(a) is a diagram showing an example of an input UI screen. The input UI screen 1510 is a screen for registering data about the child to be photographed. The UI screen 1510 has a child's name input field 1511, an age input field 1512, a personality input field 1513, a height input field 1514, a weight input field 1515, an attached photo password information input field 1516, and an attached video password information input field 1517. The UI screen 1510 also has an attach photo button 1521 and an attach video button 1522.

[0093] Each of the input fields 1511 to 1515 is an area that accepts input of data for the corresponding item by user operation. Specifically, the child's name input field 1511 allows the name of the child to be photographed to be input, and the age input field 1512 allows the age of the child to be photographed to be input.

[0094] The personality input field 1513 allows the input of a name indicating the type of personality, such as quiet, active, lonely, etc. The height input field 1514 allows the input of the height of the child to be photographed, and the weight input field 1515 allows the input of the weight of the child to be photographed. Note that the input fields 1512 to 1515 are not limited to a format for direct input, and may also be a format for inputting information selected from a pull-down menu.

[0095] The photo attachment button 1521 allows a photo of the child being photographed to be attached. When a photo of the child is attached using the photo attachment button 1521, password information is automatically entered in the password information input field 1516 for the attached photo. The video attachment button 1522 allows a video showing the child's daily activities, for example, a 1-minute or 5-minute video of the child being photographed from all angles at 30 fps, to be attached. When a video of the child is attached using the video attachment button 1522, password information is automatically entered in the password information input field 1517 for the attached video.

[0096] Although the UI screen 1510 has been described as an example in which information for all items corresponding to the input fields 1511 to 1517 is input, the present invention is not limited to this. A user may simply input the name of the child being photographed and attach a video of the child being photographed. In this case, the photographed subject recording unit 1407 estimates the personality, age, height, and weight of the child being photographed based on the attached video and individual data 317, and inputs the estimated information into the corresponding input fields.

[0097] The information entered on the UI screen 1510 is sent to the photographing object recording unit 1407 via the respective communication units 234 and 214. The photographing object recording unit 1407 converts the sent information into a format as shown in Fig. 15(b) and records it in the photographing object data 320 of the photographing object information data 1410.

[0098] 15(b) holds information corresponding to each item of a person ID 1531, a name 1532, an age 1533, a personality 1534, a height 1535, a weight 1536, an image 1537, and a video 1538. Note that the "1" in the person ID 1531 is a number assigned by the photographic subject recording unit 1407, but items other than the person ID 1531 hold information sent from the terminal device 113.

[0099] Next, the photographing object recording unit 1407 of the server device 1311 searches for records of the individual data 317 in the training data 1408 that are similar to the photographing object data 320 recorded above.

[0100] (Individual data record) 16 is a diagram showing an example record of the individual data 317 of the learning data. The individual data record 1600 holds data on each individual, including an individual ID 1601, age 1602, personality 1603, height 1604, weight 1605, and video 1606.

[0101] Individual ID 1601 indicates an ID for identifying an individual in the learning data. Age 1602, personality 1603, height 1604, weight 1605, and video 1606 are the same as the corresponding items in the photographed object data 320. When searching for records, each item other than video is compared, differences are weighted and added, and records with differences below a certain level are determined to be similar records. In this embodiment, differences in personality are weighted heavily (if there is a difference, they are not included in the similar records), and height, age, and weight are weighted lightly (even if there is a slight difference, they are included in the similar records). Note that the magnitude of the weighting is not limited to this, and weighting may be performed at any magnitude.

[0102] The photographing object recording unit 1407 of the server device 1311 sends records similar to the photographing object data 320 retrieved above to the terminal device 113 via the communication units 214 and 234. The sent data is displayed on the display / input unit 236 of the terminal device 113.

[0103] (UI screen for displaying similar records) 17A shows an example of a UI screen 1700 for displaying similar records. The UI screen 1700 is displayed on the display / input unit 236 of the terminal device 113.

[0104] The UI screen 1700 is a screen for displaying similar records, and displays a list of similar records 1710. The list 1710 displays information about each of the following items: individual ID 1711, age 1712, personality 1713, height 1714, weight 1715, and video 1716. The video 1716 item further displays buttons 1721-1723 that accept user operations to view the video of each individual. The UI screen 1700 also has an input field 1724 for inputting the individual ID of the closest individual. When one of the buttons 1721-1723 is pressed by a user operation, a video showing the normal behavior of the individual corresponding to the pressed button is played. The input field 1724 is an area that accepts user operation to input the individual that is closest to the child based on the displayed records and video.

[0105] When input on the display / input unit 236 of the terminal device 113 is completed, the user's input results are sent to the photographic object recording unit 1407 of the server device 1311 via the respective communication units 234, 214. The photographic object recording unit 1407 records the linking data between the photographic object and the individual in the photographic object and individual linking data 321 of the photographic object information data 1410.

[0106] (Data linking the photographed subject to the individual) 17(b) is a diagram showing an example of linking data between a child who is the subject of a photograph and an individual. The linking data 1730 between a child and an individual holds data for each item of a person ID 1731 and an individual ID 1732 in linkage. That is, the linking data 1730 between a child and an individual holds data in which the person ID 1731, "1," and the individual ID 1732, "5," are linked.

[0107] In this embodiment, we have described a case where a child and an individual are linked by a user's input operation, but the system may also automatically estimate the record of the individual that is most similar to the child from each video of the child and the individual, and link the person ID and individual ID based on the estimation result.

[0108] (Processing for deriving the approachable distance) Fig. 18 is a flowchart showing the overall flow of processing executed by the information processing system. Note that in Fig. 18, processing for deriving the approachable distance is executed. The unmanned mobile body 112 is in an automatic flying state, recognizing a subject to be photographed based on images obtained by photographing the surroundings of the unmanned mobile body 112, and photographing the subject by approaching or moving away from the recognized subject or circling the subject.

[0109] In S1801, the photographing subject estimation unit 1401 of the server device 111 estimates the state of the photographing subject, as in the first embodiment. That is, the photographed images captured by the imaging unit 226 of the unmanned mobile body 112 are acquired via the respective communication units 224 and 214, and the recognition unit 302 and state estimation unit 1403 each perform processing on the acquired photographed images. That is, the recognition unit 302 determines in which location (area) in the image the child of the photographing subject is located. The state estimation unit 1403 estimates the distance between the photographing subject 123 and the unmanned mobile body 112 from the camera parameters of the imaging unit 226, the size of the photographing subject in the photographed image, etc. The estimated distance between the photographing subject 123 and the unmanned mobile body 112 is used in S1805, which will be described in detail later.

[0110] (State estimation data record) FIG. 19 is a diagram showing an example of a record of state estimation data. A record 1900 of the state estimation data is data held in the state estimation data 319. In the record 1900 of the state estimation data, a state 1901 of the subject to be photographed is linked to learning data 1902 for state estimation. The state 1901 of the subject to be photographed holds states indicating behaviors, emotions, etc., such as "sleeping state," "eating state," "normal state," "crying state," "angry state," and "running around." The learning data 1902 for state estimation holds images for each state, such as "image of sleeping state," "image of eating state," "image of normal state," "image of crying state," "image of angry state," and "image of running around."

[0111] In the state estimation data record 1900, "sleeping state" and "image of sleeping state," "eating state" and "image of eating state," "normal state" and "image of normal state," and "crying state" and "image of crying state" are linked together. Also, "angry state" and "image of angry state," and "running around state" and "image of running around" are linked together. By using the state estimation data record 1900 shown in FIG. 19, it can be estimated which state the subject is in among "sleeping state," "eating state," "normal state," "crying state," "angry state," and "running around state."

[0112] In S1802, the state estimation unit 1403 of the server device 111 estimates the surrounding state of the subject being photographed. That is, the state estimation unit 1403 acquires the images captured by the imaging unit 226 of the unmanned mobile body 112 via the respective communication units 224 and 214, and performs processing of the state estimation unit 1403 on the acquired captured images. Specifically, the processing of the state estimation unit 1403 also estimates the surrounding state, in this example, whether there is anything to be thrown around the subject being photographed. The data shown in FIG. 20, which is surrounding state estimation data 1431 of the learning data 1408, is used.

[0113] (State estimation data record) 20 is a diagram showing an example of a record of state estimation data. A record 2000 of the state estimation data is data held in the surrounding state estimation data 1431. In the record 2000 of the state estimation data, a throwable object 2001 is linked to learning data 2002 for state estimation. The throwable object 2001 holds objects that children can throw, such as a "ball," a "shuriken," a "doll," and a "slingshot." The learning data 2002 for state estimation holds images of objects that children can throw, such as an "image of a ball," an "image of a shuriken," an "image of a doll," and an "image of a slingshot."

[0114] In record 2000 of the state estimation data, a "ball" is linked to an "image of a ball," a "shuriken" is linked to an "image of a shuriken," a "doll" is linked to an "image of a doll," and a "slingshot" is linked to an "image of a slingshot." By using record 2000 of the state estimation data shown in FIG. 20, the state estimation unit 1403 estimates whether there is anything that can be thrown near the subject 123 to be photographed.

[0115] In S1803, the state recording unit 1404 records in the time-series data 1409 the state of the subject estimated in S1801 and the surrounding state estimated in S1802.

[0116] (Time series data records) 21 is a diagram showing an example of a record of time-series data of the state of a subject to be photographed. A record 2100 of the time-series data of the state of a subject to be photographed is held in the time-series data of the state of the subject to be photographed (hereinafter referred to as "time-series data") 1409. The record 2100 of the time-series data of the state of a subject to be photographed associates and holds in chronological order information indicating time 2101, person ID 2102, state 2103, and surrounding state 2104.

[0117] Time 2101 indicates the time when S1801 was executed. Person ID 2102 indicates an ID for identifying a child who is the subject recognized by the recognition unit 302. State 2103 indicates the state estimated by the state estimation unit 1403. Surrounding state 2104 indicates the surrounding state estimated by the state estimation unit 1403.

[0118] In the record 2100 of time-series data of the state of the photographing object, only when the state changes are recorded, but it is also possible to record when the state does not change.

[0119] In S1804, the approachable distance calculation unit 1405 executes a process of calculating the approachable distance. The approachable distance calculation process S1804 is performed using the approachable distance data record, which is the approachable distance data 1418 of the learning data 1408. The calculation is performed by adding the condition of the first embodiment, that is, whether there is anything nearby that can be thrown, which is a surrounding state.

[0120] (Approachable distance data record) Fig. 22 is a diagram showing an example of an approachable distance data record. Fig. 22 shows data that records the approachable distance corresponding to the state change of individual ID=5. Note that Fig. 22 shows an example of a record that adds, to the same data as Fig. 11 of the first embodiment, information about the surrounding state, such as whether there is anything nearby that can be thrown. This makes it possible to calculate the approachable distance under conditions that take the surrounding state into account.

[0121] The approachable distance data record 2200 stores information on each of the following items in association with each other: No. 2201, individual ID 2202, state before change 2203, elapsed time since state before change 2204, state after change 2205, surrounding state 2206, and approachable distance 2207.

[0122] The states will be explained using record No. 3 as an example. The state before the change 2203 is "angry state", the time elapsed from the state before the change 2204 is "2 minutes", and the state after the change 2205 is "normal state". In addition, the surrounding state 2206 is "there is something to throw nearby", which indicates that the state became normal after 2 minutes had passed since the angry state. In other words, it indicates that the individual ID "5" is linked to the "woke state" (first state), the "normal state" (second state), the time it took to change from the "woke state" to the "normal state" (2 minutes), and the approachable distance "100 cm".

[0123] The approachable distance calculation process S1804 is performed by finding the record of the approachable distance data in FIG. 22 that corresponds to the time series data in FIG. 21. As an example, the approachable distance calculation for the time "2023 / 2 / 3 13:02" in FIG. 21 will be described. From the record 2100 of the time series data in FIG. 21, it can be seen that at "2023 / 2 / 3 13:00" the state was angry and there was something nearby that could be thrown, and at "2023 / 2 / 3 13:02" the state changed to normal and there was something nearby that could be thrown. Furthermore, regarding the link between person IDs and individual IDs, from the record example in FIG. 17(b), it can be seen that the person ID "1" is linked to the individual ID "5". From the above, a match is made to the record No. 3 in FIG. 22. As a result, the approachable distance is calculated to be 100 cm.

[0124] The above describes an example in which information for a time identical to a time recorded in record 2100 of the time-series data in FIG. 21 exists. However, this is not limited to this example, and the timing for executing the approachable distance calculation process may be any timing. For example, assume that the timing for executing the approachable distance calculation process is "2023 / 2 / 3 13:01." Since no state associated with this time is recorded, this indicates that the state has remained unchanged since times prior to "2023 / 2 / 3 13:01." At "2023 / 2 / 3 13:00," which is prior to "2023 / 2 / 3 13:01," an angry state is recorded, although there is something nearby. From these records, it is determined that the state has not changed for one minute, so it matches record No. 2 in record 2200 of the approachable distance data in FIG. 22. The approachable distance calculated from record No. 2 is 130 cm.

[0125] In S1805, the control instruction unit 306, similar to S804, compares the approachable distance calculated in S1804 with the distance between the subject 1322 and the unmanned mobile object 112 estimated in S1801. Then, depending on the comparison result, the control instruction unit 306 issues a control instruction to the movement control unit 227 of the unmanned mobile object 112 via the communication units 214 and 224. Note that the content of the control instruction is the same as in the first embodiment, and detailed description thereof will be omitted.

[0126] The control method for the unmanned mobile body 112 is not limited to the flow shown in FIG. 18 , and feedback of the control results may be performed. For example, when the unmanned mobile body 112 approaches the photographic target 123 and the photographic target 123 starts moving toward the unmanned mobile body 112, information indicating the distance at which the photographic target 123 started moving is sent to the photographic target recording unit 1407 via the communication units 224 and 214. This information indicating the distance is obtained in S1801 by repeatedly executing the process shown in FIG. 18 . The photographic target recording unit 1407 may store the person ID 2301, the state before change 2302, the elapsed time from the state before change 2303, the state after change 2304, the surrounding state 2305, and the approachable distance 2306 shown in FIG. 23 as difference data 1422 between the photographic target and the individual. Furthermore, when calculating the approachable distance, if data corresponding to the time-series data exists in the difference data 1422 between the photographic target and the individual, the data corresponding to the time-series data is used to calculate the approachable distance. Furthermore, the movement control unit 227 performs movement control to maintain a distance from the photographing target 123. That is, when the data corresponding to the time-series data is difference data 2300 between the photographing target and an individual, as shown in Fig. 23, 120 cm is obtained as the calculation result of the approachable distance. Then, the movement of the unmanned moving body is controlled in accordance with the approachable distance being 120 cm.

[0127] That is, when the state of the subject to be photographed differs from the state of the subject to be photographed indicated by the status information used to acquire the approachable distance information at a distance different from the approachable distance indicated by the approachable distance information, the following data may be recorded as difference data between the subject to be photographed and the individual 1422. That is, data linking the state of the subject to be photographed indicated by the status information with the distance between the unmanned moving body and the subject to be photographed may be recorded as difference data between the subject to be photographed and the individual 1422. Then, difference data between the subject to be photographed and the individual 1422 may be used when calculating the approachable distance.

[0128] As described above, according to this embodiment, even if the subject of photography is a child, the approachable distance can be derived as time-series data, as in embodiment 1. Photography can be performed while maintaining an appropriate distance from the object of photography.

[0129] Although the above describes a situation in which a ball is present around the child, this is not limiting. For example, the present invention can also be applied to a situation in which a wall is present around the child. In this case, even if the distance between the child and the unmanned moving body is longer than the calculated approachable distance, the magnitude of the approachable distance may change. For this reason, in an area where there is no wall in the direction away from the child, the approachable distance is calculated at any time interval, such as 5 seconds, and the movement of the unmanned moving body is controlled according to the calculated approachable distance.

[0130] <Other embodiments> In the above, the case where the subject to be photographed is one object (for example, one pet or one child) has been described, but the present invention is not limited to this and the subject to be photographed may be two or more objects. In this case, the two or more objects may be recognized, and the approachable distance may be estimated based on the recognition result, and the movement of the unmanned mobile body may be controlled and instructed based on the estimated approachable distance.

[0131] The above description has been given using an example of a record of approachable distance data in which the first and second states of an object, the time it takes to change from the first state to the second state, and the approachable distance are linked, but the present invention is not limited to this. For example, an example of a record of approachable distance data in which the first and second states of an object are linked to the approachable distance may be used.

[0132] The present disclosure can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The program may also be provided by recording it on a computer-readable storage medium.

[0133] The disclosure of this embodiment includes the following configuration examples. (Configuration 1) a state information acquisition means for acquiring state information indicating a state of an object to be photographed; a recording means for recording the acquired status information; distance information acquisition means for acquiring approachable distance information indicating an approachable distance to the object based on the recorded state information; a control means for controlling the movement of the unmanned moving body that photographs the object in accordance with the obtained approachable distance information; An information processing device comprising: (Configuration 2) The state information acquisition means acquires state information indicating the state of the object by estimating the state of the object. 2. The information processing device according to configuration 1, (Configuration 3) The distance information acquisition means acquires the approachable distance information by inputting the recorded state information into a trained model that has been trained to receive the state of the object as an input and output the approachable distance information. 3. The information processing device according to configuration 1 or 2. (Configuration 4) the trained model is a trained model that has been trained to receive as input a first state and a second state of the object and an elapsed time until the object changes from the first state to the second state, and to output the approachable distance information; the state information acquisition means acquires the state information indicating a first state of the object and a second state of the object; The distance information acquisition means acquires the approachable distance information by inputting, into the trained model, a first state and a second state of the object indicated by the recorded state information and an elapsed time from when the object changes from the first state to when the object changes to the second state. 4. The information processing device according to configuration 3. (Configuration 5) the state information acquisition means acquires the state information indicating a state of the object and a state of the surroundings of the object; The distance information acquisition means acquires the approachable distance information by inputting the state of the object and the state of the surroundings of the object indicated by the state information into the trained model, which has been trained to receive the state of the object and the state of the surroundings of the object as input and output the approachable distance information. 4. The information processing device according to configuration 3. (Configuration 6) the trained model is a trained model that has been trained to receive as input a first state and a second state of the object, an elapsed time from when the object changes from the first state to the second state, and a state of the surroundings of the object, and to output the approachable distance information; the state information acquisition means acquires the state information indicating a first state of the object, a second state of the object, and a state of the surroundings of the object; The distance information acquisition means acquires the approachable distance information by inputting, into the trained model, the first state and the second state of the object indicated by the recorded state information, the elapsed time from when the object changes from the first state to when the object changes to the second state, and the state of the surroundings of the object. 4. The information processing device according to configuration 3. (Configuration 7) a recognition means for recognizing the object based on a photographed image obtained by photographing the object and individual data for recognizing the object to be photographed; and The distance information acquisition means acquires the approachable distance information corresponding to the recognized object. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 8) If the subject is a pet, The information processing device according to any one of configurations 1 to 7, wherein the individual data includes at least one of the breed of the subject, the age of the subject, the personality of the subject, the body length of the subject, the weight of the subject, and a video image of the subject. (Configuration 9) If the subject is a child, The individual data includes at least one of the age of the subject, the personality of the subject, the height of the subject, the weight of the subject, and a video image of the subject. 9. The information processing device according to any one of configurations 1 to 8. (Configuration 10) the state information acquisition means further acquires distance information indicating a distance between the object and the unmanned moving body; The control means controls the movement of the unmanned moving body based on the distance indicated by the acquired distance information and the distance indicated by the approachable distance information. 10. The information processing device according to any one of configurations 1 to 9. (Configuration 11) The control means If the distance indicated by the acquired distance information is shorter than the distance indicated by the approachable distance information, the movement of the unmanned moving body is controlled in a direction away from the object. 11. The information processing device according to configuration 10. (Configuration 12) The control means If the distance indicated by the acquired distance information is longer than the distance indicated by the approachable distance information, the movement of the unmanned moving body is controlled in a direction approaching the object. 12. The information processing device according to configuration 10 or 11. (Configuration 13) When the state of the object is different from the state of the object indicated by the state information used to acquire the approachable distance information at a distance different from the approachable distance indicated by the approachable distance information, the recording means records data linking the state of the object indicated by the state information with the distance between the unmanned moving body and the object. 13. The information processing device according to any one of configurations 1 to 12. (Configuration 14) The distance information acquisition means acquires the approachable distance information from the recorded data and a trained model that has been trained to input the state of the object and output the approachable distance information. 14. The information processing device according to configuration 13. (Configuration 15) a state information acquisition step of acquiring state information indicating a state of an object to be photographed; a recording step of recording the acquired status information; a distance information acquisition step of acquiring approachable distance information indicating an approachable distance to the object based on the recorded state information; a control step of controlling movement of the unmanned moving body that photographs the object in accordance with the obtained approachable distance information; An information processing method comprising: (Configuration 16) 16. A program for causing a computer to execute the information processing method according to claim 15.

Claims

1. a state information acquisition means for acquiring state information indicating a state of an object to be photographed; a recording means for recording the acquired status information; distance information acquisition means for acquiring approachable distance information indicating an approachable distance to the object based on the recorded state information; a control means for controlling the movement of the unmanned moving body that photographs the object in accordance with the obtained approachable distance information; An information processing device comprising:

2. The state information acquisition means acquires state information indicating the state of the object by estimating the state of the object.

2. The information processing apparatus according to claim 1, wherein:

3. The distance information acquisition means acquires the approachable distance information by inputting the recorded state information into a trained model that has been trained to receive the state of the object as an input and output the approachable distance information.

2. The information processing apparatus according to claim 1, wherein:

4. the trained model is a trained model that has been trained to receive as input a first state and a second state of the object and an elapsed time until the object changes from the first state to the second state, and to output the approachable distance information; the state information acquisition means acquires the state information indicating a first state of the object and a second state of the object; The distance information acquisition means acquires the approachable distance information by inputting, into the trained model, a first state and a second state of the object indicated by the recorded state information and an elapsed time from when the object changes from the first state to when the object changes to the second state.

4. The information processing apparatus according to claim 3,

5. the state information acquisition means acquires the state information indicating a state of the object and a state of the surroundings of the object; The distance information acquisition means acquires the approachable distance information by inputting the state of the object and the state of the surroundings of the object indicated by the state information into the trained model, which has been trained to receive the state of the object and the state of the surroundings of the object as input and output the approachable distance information.

4. The information processing apparatus according to claim 3,

6. the trained model is a trained model that has been trained to receive as input a first state and a second state of the object, an elapsed time from when the object changes from the first state to the second state, and a state of the surroundings of the object, and to output the approachable distance information; the state information acquisition means acquires the state information indicating a first state of the object, a second state of the object, and a state of the surroundings of the object; The distance information acquisition means acquires the approachable distance information by inputting, into the trained model, the first state and the second state of the object indicated by the recorded state information, the elapsed time from when the object changes from the first state to when the object changes to the second state, and the state of the surroundings of the object.

4. The information processing apparatus according to claim 3,

7. a recognition means for recognizing the object based on a photographed image obtained by photographing the object and individual data for recognizing the object to be photographed; and The distance information acquisition means acquires the approachable distance information corresponding to the recognized object.

2. The information processing apparatus according to claim 1, wherein:

8. If the subject is a pet, 8. The information processing device according to claim 7, wherein the individual data includes at least one of the breed of the subject, the age of the subject, the personality of the subject, the body length of the subject, the weight of the subject, and a video image of the subject.

9. If the subject is a child, The individual data includes at least one of the age of the subject, the personality of the subject, the height of the subject, the weight of the subject, and a video image of the subject.

8. The information processing apparatus according to claim 7,

10. the state information acquisition means further acquires distance information indicating a distance between the object and the unmanned moving body; The control means controls the movement of the unmanned moving body based on the distance indicated by the acquired distance information and the distance indicated by the approachable distance information.

2. The information processing apparatus according to claim 1, wherein:

11. The control means If the distance indicated by the acquired distance information is shorter than the distance indicated by the approachable distance information, the movement of the unmanned moving body is controlled in a direction away from the object.

11. The information processing apparatus according to claim 10,

12. The control means If the distance indicated by the acquired distance information is longer than the distance indicated by the approachable distance information, the movement of the unmanned moving body is controlled in a direction approaching the object.

11. The information processing apparatus according to claim 10,

13. When the state of the object is different from the state of the object indicated by the state information used to acquire the approachable distance information at a distance different from the approachable distance indicated by the approachable distance information, the recording means records data linking the state of the object indicated by the state information with the distance between the unmanned moving body and the object.

2. The information processing apparatus according to claim 1, wherein:

14. The distance information acquisition means acquires the approachable distance information from the recorded data and a trained model that has been trained to input the state of the object and output the approachable distance information.

14. The information processing apparatus according to claim 13,

15. a state information acquisition step of acquiring state information indicating a state of an object to be photographed; a recording step of recording the acquired status information; a distance information acquisition step of acquiring approachable distance information indicating an approachable distance to the object based on the recorded state information; a control step of controlling movement of the unmanned moving body that photographs the object in accordance with the obtained approachable distance information; An information processing method comprising:

16. A program for causing a computer to execute the information processing method according to claim 15.

Images