Information processing method, information processing device, and program

Abstract

Provided is an information processing method in which sensing information obtained by sensing the surroundings of a first moving body that autonomously moves is obtained (S301), a second moving body existing in the surroundings of the first moving body is detected on the basis of the sensing information (S302), and the second moving body is detected on the basis of the detected second moving body. When it is detected that there are two or more second moving bodies present around the first moving body, it is determined, on the basis of the sensing information, whether or not the two or more second moving bodies are in a state having a predetermined correlation (S305), and when the two or more second moving bodies are in a state having a predetermined correlation (YES in S305), the second moving bodies are in a state having a predetermined correlation (YES in S305). The predetermined work is performed so as to avoid contact between the two or more second moving bodies and the first moving body (e.g., S306).

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing device, and a program. Background Art

[0002] In order to improve the safety when driving a vehicle, a method has been proposed. In this method, mobile objects such as pedestrians, bicycles, wheelchairs, baby carriages, pets, etc. around the vehicle are detected, the risk of the detected mobile objects relative to the vehicle is calculated, and the parameters or medium of the unidentified notification information are changed according to the calculated risk (for example, patent document 1).

[0003] In Patent Document 1, when the moving object around the vehicle is a person, even when the person is not recognized by the vehicle, attention can be called to the person.

[0004] (Prior art literature)

[0005] (Patent Document)

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-149309 Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, in the method of Patent Document 1, even in the case where the pedestrian is a parent and a child (parent and child) with the parent holding the child's hand and there are pedestrians around the vehicle, the risk is calculated separately for the pedestrian as a parent and the pedestrian as a child. For example, in the case where a parent is holding the child's hand, the risk to the pedestrian as a child is considered to be lower than the risk to the pedestrian as a child alone. That is, in the method of Patent Document 1, when the moving body belongs to a group, the change in risk caused by the influence of other members in the group is not considered. As a result, if the risk is overestimated, unnecessary control will be implemented on the vehicle, resulting in a decrease in the efficiency of the vehicle's operation. And, if the risk is underestimated, the vehicle will not be adequately controlled, resulting in the moving body falling into a dangerous state.

[0009] In view of the above circumstances, an object of the present disclosure is to provide an information processing method and the like that can perform necessary and sufficient control on a mobile object such as a vehicle.

[0010] Means for solving problems

[0011] An information processing method involved in one embodiment of the present disclosure is an information processing method in an information processing device. In the information processing method, sensing information is obtained, and the sensing information is information obtained by sensing the surroundings of a first moving body that is moving autonomously. A second moving body that exists around the first moving body is detected based on the sensing information. When it is detected that there are two or more second moving bodies that exist around the first moving body, it is determined based on the sensing information whether the two or more second moving bodies are in a state with a specified correlation. When the two or more second moving bodies are in a state with the specified correlation, a specified operation is performed to prevent the two or more second moving bodies from contacting the first moving body.

[0012] In addition, these general or specific aspects may be implemented by a system, method, integrated circuit, computer program, or a computer-readable CD-ROM or other recording medium, or by any combination of a system, method, integrated circuit, computer program, and recording medium. Furthermore, the recording medium may be a non-transitory recording medium.

[0013] Effects of the Invention

[0014] The information processing method of the present disclosure can perform necessary and sufficient control on a moving object such as a vehicle.

[0015] In addition, better effects and advantages of one embodiment of the present disclosure can be clearly seen from the specification and the drawings. Although the relevant advantages and / or effects are provided by the features described in several embodiments, the specification and the drawings, it is not necessary to provide all of them in order to obtain one or more of the same features. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 An example of the overall configuration including the information processing device according to the embodiment is shown.

[0017] Figure 2 This is a block diagram showing an example of the configuration of a first moving object according to the embodiment.

[0018] Figure 3 This is a block diagram showing an example of the configuration of a terminal according to an embodiment.

[0019] Figure 4 This is a block diagram showing an example of the configuration of an information processing device according to an embodiment.

[0020] Figure 5A An example of the risk value according to the attribute of the second moving object according to the embodiment is shown.

[0021] Figure 5B An example of a risk value according to the distance between the first moving object and the second moving object according to the embodiment is shown.

[0022] Figure 5C An example of the risk value according to the speed of the first moving object according to the embodiment is shown.

[0023] Figure 5D An example of the risk value according to the speed of the second moving object according to the embodiment is shown.

[0024] Figure 6 An example of attributes used for calculation of individual risk according to the embodiment is shown.

[0025] Figure 7 This is a diagram showing a conditional flow as an example of group determination information according to the embodiment.

[0026] Fig. 8A An example in which two second moving objects according to the embodiment have a predetermined relationship is conceptually shown.

[0027] Figure 8B An example in which two second moving objects according to the embodiment have a predetermined relationship is conceptually shown.

[0028] Figure 8C An example in which two second moving objects according to the embodiment have a predetermined relationship is conceptually shown.

[0029] Fig.8D An example in which two second moving objects according to the embodiment have a predetermined relationship is conceptually shown.

[0030] Fig. 8E An example in which two second moving objects according to the embodiment have a predetermined relationship is conceptually shown.

[0031] Fig. 9 An example of conditions for determining whether or not a predetermined correlation exists according to the embodiment is shown.

[0032] Fig.10 An example of the risk correction value according to the embodiment is shown.

[0033] Fig.11 An example of risk correction based on the influence from other second moving objects in the group according to the embodiment is conceptually shown.

[0034] Fig.12An example of risk correction based on the influence from other second moving objects in the group according to the embodiment is conceptually shown.

[0035] Fig.13 An example of risk correction based on the influence from other second moving objects in the group according to the embodiment is conceptually shown.

[0036] Fig.14 An example of risk correction based on the influence from other second moving objects in the group according to the embodiment is conceptually shown.

[0037] Fig.15 An example of information generated according to the corrected individual risk value according to the embodiment is shown.

[0038] Fig.16 It is a sequence diagram showing the operation including the overall configuration of the information processing device involved in the embodiment.

[0039] Fig.17 This is a flowchart showing an example of the operation of the information processing device according to the embodiment.

[0040] Fig.18 It is shown Fig.17 A flowchart of an example of the details of step S307 is shown.

[0041] Fig.19A The risk calculated by the method according to the comparative example in a certain scenario is shown.

[0042] Fig.19B The risk calculated by the method involved in the present disclosure in a certain scenario is shown.

[0043] Fig. 20 An example of risk correction based on the influence from other second moving objects in the group according to the modification of the embodiment is conceptually shown.

[0044] Fig.21 This is a diagram for explaining control of a first moving object when two second moving objects having a predetermined correlation avoid an obstacle according to a modification of the embodiment.

[0045] Fig. 22 This is a diagram for explaining an example of risk correction based on the positional relationship between the second mobile object and the first mobile object in a group according to a modification of the embodiment. DETAILED DESCRIPTION

[0046] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0047] In addition, the embodiments described below are all general or specific examples. The numerical values, shapes, components, configuration positions and connection methods of the components, steps and the order of the steps shown in the following embodiments are all examples, and their purpose is not to limit the present disclosure. Moreover, the components of the following embodiments that are not recorded in the independent technical solutions will be described as arbitrary components.

[0048] In addition, each figure is a schematic diagram and is not a strict illustration. In addition, in each figure, the same components are given the same reference numerals.

[0049] (Implementation Method)

[0050] Hereinafter, an information processing method and an information processing device according to the present embodiment will be described.

[0051] [1 Overall composition]

[0052] Figure 1 An example of the overall configuration including the information processing device 10 according to the present embodiment is shown.

[0053] Figure 1 The overall structure shown includes an information processing device 10, a terminal D1 used by a remote monitor, and a first mobile body M1, which are connected via a network N so as to be able to communicate with each other. The remote monitor is a person who remotely monitors and remotely controls the first mobile body M1 through the terminal D1. In other words, Figure 1 The overall structure shown is an information processing system for a remote monitor to monitor the first mobile object M1 and the information processing device 10, and to remotely monitor or remotely control the first mobile object M1 according to information notified from the terminal D1.

[0054] The network N is a predetermined wired or wireless communication network. Although the network N is, for example, a mobile communication network for a cellular phone, a satellite communication network, a wide area network using Wi-Fi, etc., the network N is not limited to these. The information processing device 10, the terminal D1, and the first mobile body M1 may be physically located anywhere as long as they are connected to the network N so as to be able to communicate.

[0055] In addition, in this embodiment, if Figure 1 As shown, although the overall structure is described as having one first moving body M1 and one terminal D1, it is not limited to this. It can also have more than two first moving bodies M1 and more than two terminals D1, and the terminal D1 and the information processing device 10 can also be integrated. And it can also be, Figure 1The overall configuration shown does not include the terminal D1 , but is constituted only of the information processing device 10 and the first mobile body M1 .

[0056] [1.1 First moving body M1]

[0057] The first mobile body M1 is a mobile robot, vehicle, etc. that moves autonomously, and at least one of the mobile robots and vehicles that can be remotely operated and remotely monitored by a monitor. The first mobile body M1 is not limited to a mobile robot or a vehicle, and may also be any one of an AGV (Automated Guided Vehicle), a personal mobile device, an airplane, a drone, a ship, a mobile robot, a construction equipment, an agricultural machine (tractor), etc. In the case where the first mobile body M1 is a vehicle, it may or may not carry people. In addition, in the case where the first mobile body M1 is a vehicle, it may be an autonomous driving vehicle or a non-autonomous driving vehicle driven by a passenger. In the present embodiment, there is a situation where the first mobile body M1 controls the steering angle, acceleration and deceleration, etc. according to the processing results performed by the information processing device 10 or the input content input by the remote monitor to the terminal D1.

[0058] Figure 2 This is a block diagram showing an example of the configuration of the first moving body M1 according to the present embodiment.

[0059] like Figure 2 As shown, the first moving body M1 includes a communication device M11, a sensing device M12, a state information obtaining device M13, a control device M14, and a driving device M15.

[0060] The communication device M11 is, for example, a communication interface and can be connected to the network N. The communication device M11 is communicatively connected to the information processing device 10 via the network N, and as described later, periodically transmits state information and sensing information to the information processing device 10 .

[0061] The sensing device M12 is, for example, a camera, a LiDAR (Light Detection And Ranging) device, etc., and obtains sensing information by photographing the surroundings of the first moving body M1. The sensing information may be, for example, image information obtained by photographing the surroundings of the first moving body M1, or may be video information including one or more image information obtained by photographing the surroundings of the first moving body M1. Furthermore, the sensing device M12 obtains the sensing information at a predetermined interval, i.e., periodically. In addition, photographing is an example of sensing.

[0062] The state information acquisition device M13 is equipped with various sensors, and acquires state information related to the moving state of the first moving body M1 by detecting the state information related to the moving state of the first moving body M1. Here, the state information only needs to include the moving speed, moving direction and moving state of the first moving body M1. The moving state, for example, indicates that the first moving body M1 is in one of the states of traveling, stopping or temporarily stopping. As various sensors, GPS (Global Positioning System) can also be included.

[0063] Alternatively, the status information may further include the position information of the first mobile body M1, the speed, acceleration, jerk, tilt, azimuth, steering angle, remaining fuel, operating status of the direction indicator, etc. of the first mobile body M1. Furthermore, the status information may also include the operating status of ABS (Anti-lock Braking System), the operating status of AEB (Automatic Emergency Braking), etc. Furthermore, the status information may also include the presence or absence of surrounding objects of the first mobile body M1, the number and type of the surrounding objects, the distance between the surrounding objects, the speed difference, the weather conditions, the surrounding sound information, the acceleration difference, the angle at which the sensing device M12 is installed on the first mobile body M1, and the position information, etc. The status information may also include the above-mentioned various information, also referred to as the type of status information. That is, the status information itself may have multiple types.

[0064] Furthermore, the state information obtaining device M13 obtains state information at predetermined intervals, that is, regularly, using various sensors.

[0065] The control device M14 includes a computer, and uses the computer to control the speed and steering of the drive device M15. The control device M14 can use the computer to control the speed and steering of the drive device M15 according to the operation of the passenger or the operation of the remote monitor.

[0066] The driving device M15 is composed of a driving body such as wheels of the first moving body M1, a motor, an engine, and the like, and is controlled by the control device M14.

[0067] [1.2 Terminal D1]

[0068] The terminal D1 can check the status information of the first moving object M1 or the processing result processed by the information processing device 10. In addition, the terminal D1 may control the information processing device 10 and the first moving object M1 according to the input content of the remote monitor.

[0069] Figure 3 This is a block diagram showing an example of the configuration of the terminal D1 according to the present embodiment.

[0070] like Figure 3 As shown, the terminal D1 includes a communication device D11, an input device D12, and an output device D13.

[0071] The communication device D11 is, for example, a communication interface, and can be connected to the network N. The communication device D11 is connected to the information processing device 10 or the first mobile body M1 via the network N so as to be communicable.

[0072] The input device D12 is composed of, for example, a joystick, a touch screen, a button, etc., and is used by a remote monitor to input input information. The input information is information about the control of the acceleration and deceleration of the first moving body M1 and the control of the steering angle via the network N, or information about the set value when the information processing device 10 performs information processing.

[0073] The output device D13 is composed of, for example, a display, a speaker, a lamp, etc., and is controlled according to the notification information obtained from the information processing device 10 and the first mobile body M1. The notification information may include, for example, the operation information of the information processing device 10 and the first mobile body M1, and instruction information for displaying the operation information. In addition, the notification information is not limited to the case where it includes instruction information for displaying the operation information, and may also include instruction information for flashing a target object on the screen of the display, activating a button, emitting a sound, etc.

[0074] [1.3 Information processing device 10]

[0075] The information processing device 10 is, for example, a server, and executes the information processing method according to the present embodiment.

[0076] Figure 4 This is a block diagram showing an example of the configuration of the information processing device 10 according to the present embodiment.

[0077] like Figure 4 As shown, the information processing device 10 includes a communication unit 11 , a storage unit 12 , and a processing unit 20 .

[0078] The communication unit 11 is, for example, a communication interface, and can be connected to the network N. The communication unit 11 is connected to the first mobile body M1 and the terminal D1 via the network N so as to be able to communicate. The communication unit 11 obtains state information related to the movement state of the first mobile body M1 that performs autonomous movement, and sensing information obtained by photographing the surroundings of the first mobile body M1. The communication unit 11 is an example of an acquisition unit of the present disclosure.

[0079] In this embodiment, the communication unit 11 receives input information sent from the terminal D1, and state information and sensing information sent from the first mobile body M1. Alternatively, the communication unit 11 further obtains environmental information including information about the road on which the first mobile body M1 moves, traffic conditions on the road, and information about terrain related to the road, and causes the storage unit 12 to store the environmental information.

[0080] The communication unit 11 transmits the notification information generated by the processing unit 20 to the terminal D1, and transmits the control information generated by the processing unit 20 to the first moving object M1 via the terminal D1. Thus, the terminal D1 and the first moving object M1 are controlled by the notification information and the control information.

[0081] The storage unit 12 is a database for storing information used for processing in the processing unit 20 , and is implemented by a HDD (Hard Disk Drive) or an SSD (Solid State Drive) or the like.

[0082] In this embodiment, the storage unit 12 stores information required for detecting and determining the position of the second moving body. The required information is, for example, a data group for image recognition, a setting value for converting coordinates on an image into position coordinates, etc. In addition, the storage unit 12 stores information for calculating individual risks, group determination information for determining whether two or more second moving bodies belong to a group, and information that associates risks with control contents.

[0083] In addition, the storage unit 12 will be described later.

[0084] [1.3.1 Processing unit 20]

[0085] The processing unit 20 generates notification information for the terminal D1 and control information for the first mobile object M1 based on the information obtained from the communication unit 11 and the storage unit 12 .

[0086] like Figure 4 As shown, the processing unit 20 includes a second mobile body detection unit 21, a second mobile body position determination unit 22, an individual risk calculation unit 23, a group relationship determination unit 24, a risk correction unit 25, an information generation unit 26, and an output unit 27. The processing unit 20 includes, for example, a computer including a memory and a processor (microprocessor), and the processor executes a predetermined program stored in the memory to realize various functions of each component.

[0087] [1.3.2 Second moving object detection unit 21]

[0088] The second moving object detection unit 21 detects two or more second moving objects existing around the first moving object M1 and specifies the attributes of each of the two or more second moving objects based on the acquired sensing information.

[0089] The second moving body detection unit 21 can realize functions such as detection function and attribute determination function by executing a predetermined program stored in the memory by the processor. Here, the second moving body is equivalent to a biological body or a passenger vehicle, pedestrian, bicycle, wheelchair, baby carriage, electric wheelchair for the elderly, pet, etc. that may contact or collide with the first moving body M1. In addition, the second moving body is different from the first moving body M1, that is, it is not a vehicle, a mobile robot, etc.

[0090] In the present embodiment, the second moving body detection unit 21 obtains information required for detecting the second moving body from the storage unit 12, and detects the second moving body using the obtained required information and, for example, image information of the first moving body M1 obtained by the communication unit 11. Furthermore, the second moving body detection unit 21 determines the attributes of each second moving body based on, for example, image information of the first moving body M1 obtained by the communication unit 11. Here, the attributes of the second moving body include age, gender, height, clothing, direction, posture, and the like. In addition, the second moving body detection unit 21 can detect the second moving body and determine the attributes of the second moving body by executing a prescribed program stored in the memory by the processor, such as by using machine learning based on deep learning or other AI technologies to perform image recognition.

[0091] [1.3.3 Second moving object position determination unit 22]

[0092] The second moving object position identifying unit 22 identifies the positions of the two or more second moving objects detected by the second moving object detecting unit 21. The second moving object position identifying unit 22 can implement functions such as a position identifying function by executing a predetermined program stored in a memory through a processor.

[0093] In this embodiment, the second mobile body position determination unit 22 determines the coordinates of the second mobile body detected by the second mobile body position determination unit 22 on the image. The second mobile body position determination unit 22 obtains information required for determining the position of the first mobile body M1, such as the height of the camera, and converts the position coordinates of the second mobile body detected on the image into the position coordinates of the second mobile body relative to the first mobile body M1. In this way, the second mobile body position determination unit 22 determines the position of the second mobile body detected by the second mobile body detection unit 21 relative to the first mobile body M1.

[0094] Alternatively, when the state information includes the position coordinates, direction, camera position information, camera tilt, camera viewing angle, etc. of the first mobile body M1, the second mobile body position determination unit 22 calculates the position coordinates of the second mobile body using the information included in the state information. Furthermore, the second mobile body position determination unit 22 further calculates the speed information and acceleration information of the second mobile body relative to the first mobile body M1 based on the calculated time change of the position information of the second mobile body.

[0095] [1.3.4 Individual risk calculation department 23]

[0096] The individual risk calculation unit 23 calculates individual risks indicating the possibility of danger of two or more second mobile bodies contacting the first mobile body M1, respectively, based on the state information obtained by the communication unit 11 and the position determined by the second mobile body detection unit 21. The individual risk calculation unit 23 can realize the calculation function by executing a predetermined program stored in the memory by the processor. In addition, the individual risk calculation unit 23 only needs to calculate the individual risk based on at least the position of the first mobile body M1 and the respective positions of the two or more second mobile bodies (for example, the positional relationship between the first mobile body M1 and the respective positions of the two or more second mobile bodies).

[0097] In this embodiment, the individual risk calculation unit 23 can calculate by comparing the attributes of each second moving object specified by the second moving object detection unit 21, the position information, speed information, and acceleration information obtained by the second moving object position determination unit 22 with the reference information for risk calculation.

[0098] Here, reference information used for risk calculation will be described.

[0099] Figure 5A An example of the risk value according to the attribute of the second moving object according to the present embodiment is shown. Figure 5A Table T1 shows an example of risk values ​​corresponding to age attributes when the second moving object is a pedestrian, a wheelchair, a pet, or a bicycle. Figure 5A In the example shown, the risk value differs depending on whether the second moving object is older than 9 years old or younger than 9 years old, and here, 9 years old is an age that is difficult to distinguish.

[0100] Figure 5B An example of the risk value according to the distance between the first moving object M1 and the second moving object according to the present embodiment is shown. Figure 5B Table T2 shows an example in which the risk value increases as the distance between the first moving object M1 and the second moving object decreases. Figure 5C An example of the risk value according to the speed of the first moving body M1 according to the present embodiment is shown. Figure 5C Table T3 shows an example in which the risk value increases as the speed of the first moving body M1 increases. Figure 5D An example of the risk value according to the speed of the second moving object according to the present embodiment is shown. Figure 5D Table T4 shows an example in which the risk value increases as the speed of the second moving object increases.

[0101] In this embodiment, the risk indicates the probability of the second mobile body colliding with the first mobile body M1 such as a vehicle, and may be a numerical value. The individual risk indicates the probability of the danger of two or more second mobile bodies (individually) contacting (colliding) with the first mobile body M1. In addition, the risk may be calculated by combining information showing a plurality of risk values. That is, the individual risk calculation unit 23 may also calculate the risk of the second mobile body M1 based on the determined attributes of the second mobile body and the determined position coordinates. Figure 5A Table T1, Figure 5B Table T2, Figure 5C Table T3 and Figure 5D The risk value shown in Table T4 is determined. In this way, the individual risk calculation unit 23 can also calculate the individual risk by combining the determined risk values. As a method of combining the determined risk values, for example, addition, subtraction, multiplication or division can be used.

[0102] In addition, when using Figure 5A Table T1 to Figure 5D If the individual risk is calculated using Table T4, then Figure 5A Table T1 to Figure 5D The table T4 is stored in the storage unit 12 .

[0103] Furthermore, the attributes used by the individual risk calculation unit 23 when calculating the individual risk are not Figure 5A For example, you can also make Figure 6 Elements of these attributes are shown in which more than one combination of attributes is used in the calculation of the individual risk. Figure 6 Attribute examples (examples of elements constituting attributes) used for calculation of individual risks according to the present embodiment are shown.

[0104] An example of risk tendency based on the attributes of pedestrians as a premise for individual risk calculation is described below. For example, the tendency set in the attribute of age is that children are more likely to be interested in the first moving object M1 and the risk of approaching the first moving object M1 is higher than that of adults. The tendency set in the attribute of gender is that boys are more likely to approach the first moving object M1 than girls. The tendency set in the attribute of clothing is that, for example, uniforms and private clothes are used as elements, and if the pedestrian is wearing a uniform, the risk of approaching the first moving object M1 in order to give priority to going to and from school is low. The tendency set in the attribute of face is that the line of sight or expression is used as an element, for example, if the line of sight is directed to the first moving object M1, it indicates that the risk of being interested in the first moving object M1 and approaching the first moving object M1 is low. The tendency set in the attribute of state is that the hand is held or held as an element, for example, if the hand is held or held, the risk of approaching the first moving object M1 is low. The tendency set in the attributes of moving speed and direction is that the faster the speed is, the greater the risk of approaching the first moving object M1. The tendency set in other attributes is that when there is an obstacle such as a fence between pedestrians, the risk of approaching the first moving object M1 is small, and when a conversation is being conducted in a loud voice or a conversation about the first moving object M1 is in progress, the risk of approaching the first moving object M1 is small.

[0105] Alternatively, the individual risk calculation unit 23 may further use environmental information to calculate the individual risk. For example, it is assumed that the individual risk calculation unit 23 can determine that a plurality of pedestrians as the detected second moving bodies are present under the cliff based on the terrain information. In this case, the individual risk calculation unit 23 may also calculate the individual risk by determining a risk value indicating that the risk of approaching the first moving body M1 is small and combining it with other determined risk values.

[0106] [1.3.5 Group relationship determination unit 24]

[0107] The group relationship determination unit 24 determines whether the two or more second mobile bodies are in a state of having a predetermined association based on the determined positions and attributes. The group relationship determination unit 24 determines whether the two or more second mobile bodies are in a state of having a predetermined association based on, for example, at least one of the group determination information of the distance, line of sight, physical connection, possession, action, age, and voice between the two or more second mobile bodies. When the plurality of second mobile bodies have a predetermined association, it can be determined that the plurality of second mobile bodies belong to a group.

[0108] The group relationship determination unit 24 can realize the above-mentioned determination function by causing the processor to execute a predetermined program stored in the memory.

[0109] In the present embodiment, when the second mobile body detection unit 21 detects a plurality of second mobile bodies, the group relationship judgment unit 24 obtains the group judgment information stored in the storage unit 12. The group relationship judgment unit 24 uses the obtained group judgment information to judge whether the plurality of second mobile bodies have a prescribed association. For example, when a plurality of second mobile bodies are detected, the group relationship judgment unit 24 first determines the group formed by the plurality of second mobile bodies detected and the persons who make up the group. Then, the group relationship judgment unit 24 may also use the obtained group judgment information to judge whether the plurality of second mobile bodies have a prescribed association based on the attributes of the plurality of second mobile bodies detected, the position information relative to the first mobile body M1, and the like.

[0110] Figure 7 This is a diagram showing a conditional flow as an example of group determination information according to the present embodiment.

[0111] Here, the group judgment information includes conditions (set values) such as time and distance indicating a predetermined correlation. Figure 7 In this case, the group relationship determination unit 24 can determine the information indicating the condition of the predetermined association in the form of conditional branching as shown in the conditional flow. Figure 7 The conditional flow shown in FIG. 1 is used to determine whether the plurality of second moving bodies have a prescribed correlation. Figure 7 In the example shown, the group relationship determination unit 24 can determine that a plurality of second moving bodies have a predetermined correlation and belong to a group, for example, when the detected second moving bodies are within 2 m of each other for more than a certain period of time. Similarly, the group relationship determination unit 24 can also determine that a plurality of second moving bodies have a predetermined correlation, for example, when the second moving bodies are within 3 m of each other and move in the same direction at the same speed, when they are physically connected, when their sight lines intersect for a certain period of time, or when they are talking. Alternatively, for example, the group relationship determination unit 24 can determine whether the second moving bodies are in a state of having a predetermined correlation, that is, whether the second moving bodies belong to the same group, based at least on a positional relationship such as the second moving bodies being within 2 m of each other.

[0112] Here, using FIG. 8A to FIG. 8E A case where it can be determined that a plurality of second moving objects have a predetermined correlation will be conceptually described.

[0113] FIG. 8A to FIG. 8E An example in which two second moving objects according to the present embodiment have a predetermined relationship is conceptually shown.

[0114] Fig. 8AThe example shown is a case where the second moving body A1 and the second moving body A2, both of which are adult pedestrians, are stationary within a certain distance. Fig. 8A In the case of the relationship shown, the group relationship determination unit 24 can determine that the second mobile object A1 and the second mobile object A2 have a predetermined relationship and belong to a group.

[0115] Figure 8B The example shown is a case where the second moving body A1 and the second moving body A2, both adult pedestrians, are within a certain distance and are moving in the same direction (travel direction) at a certain speed. Figure 8B In the case of the relationship shown, the group relationship determination unit 24 can determine that the second mobile object A1 and the second mobile object A2 have a predetermined relationship and belong to a group.

[0116] Likewise, Figure 8C The example shown is a case where the second moving object A1 and the second moving object A2, both of which are adult pedestrians, are physically connected such as holding hands. Figure 8C In the case of the relationship shown, the group relationship determination unit 24 can determine that the second mobile object A1 and the second mobile object A2 have a predetermined relationship and belong to a group.

[0117] Fig.8D The example shown is a case where one of the second moving body A1 and the second moving body A2, both of which are adult pedestrians (the second moving body A1 in the figure) watches over the other (the second moving body A2 in the figure). Fig.8D In the case of the relationship shown, the group relationship determination unit 24 can determine that the second mobile object A1 and the second mobile object A2 have a predetermined relationship and belong to a group.

[0118] Fig. 8E The example shown is a case where the second moving body A1 and the second moving body A2, both of which are adult pedestrians, have their sight lines intersecting or are talking. Fig. 8E In the case of the relationship shown, the group relationship determination unit 24 can determine that the second mobile object A1 and the second mobile object A2 have a predetermined relationship and belong to a group.

[0119] In addition, the condition that can determine whether a plurality of second moving objects have a predetermined correlation is not subject to Figure 7 , FIG. 8A to FIG. 8E For example, it can also be limited to the conditions of the description. Fig. 9The conditions are determined by combining the elements shown in the figure. Fig. 9 An example of conditions for determining whether or not a predetermined correlation exists according to the present embodiment is shown.

[0120] That is, Fig. 9 As shown, when the distance between the second moving bodies is a condition, for example, elements such as contact, within 2m, and more than 2m apart can be thought of. And, when the shape of multiple second moving bodies is a condition, elements such as neat arrangement and disorder can be thought of. And, when the line of sight of multiple second moving bodies is a condition, elements such as looking at each other, only one party is guarding with line of sight, and lines of sight are in different directions can be thought of. Similarly, when the actions of multiple second moving bodies are a condition, elements such as standing and talking, walking, and running around can be thought of. And, when other conditions are taken, for example, elements such as information showing the relationship between members, the same clothing or possessions, and walking together can be thought of. That is, the group relationship judgment unit 24 can also combine these elements to determine whether there is a specified relationship based on whether the conditions composed of the combined elements are met.

[0121] In addition, the group relationship judgment unit 24 may not only judge that the two or more second moving bodies are in a state of having a prescribed association and belonging to a group, but also determine the state within the group. For example, the group relationship judgment unit 24 may determine the ratio of the number of adults and children in the group composed of the two or more second moving bodies while judging that the two or more second moving bodies belong to the group. Accordingly, in the risk correction unit 25 described later, if the number of children corresponding to each adult is smaller, the individual risk is corrected in a manner that makes the individual risk smaller, so that a more accurate risk can be calculated for each of the two or more second moving bodies. And for example, the group relationship judgment unit 24 may determine whether the group composed of the two or more second moving bodies is arranged in an orderly manner while judging that the two or more second moving bodies belong to the group. The reason is that the situation of walking in an orderly manner is often the situation of going to and from school and the destination / arrival time is fixed, and accordingly, the individual risk can be corrected in a manner that makes the individual risk smaller. And for example, the group relationship judgment unit 24 may determine whether there are physically restrained members in the group consisting of the two or more second mobile bodies while judging that the two or more second mobile bodies belong to the group. The reason is that in the case of holding hands or walking hand in hand, correction can be made in a way that the individual risk is reduced. And for example, the group relationship judgment unit 24 may determine the situation of conversation in the group consisting of the two or more second mobile bodies while judging that the two or more second mobile bodies belong to the group. The reason is that in the case of loud noise or when the topic is about vehicles, correction should be made in a way that the individual risk is increased.

[0122] Furthermore, the group relationship determination unit 24 may further use the environmental information to determine whether the two or more second moving bodies are in a state with a predetermined correlation. For example, the group relationship determination unit 24 may determine that the detected plurality of second moving bodies are students walking in a neat row and on their way to and from school, based on the time, terrain information, determined positions, and attributes included in the environmental information. Based on this, the risk correction unit 25 described later may correct the individual risk in a manner that reduces the individual risk, and calculate a more accurate risk for each of the two or more second moving bodies.

[0123] [1.3.6 Risk Correction Department 25]

[0124] The risk correction unit 25 corrects the individual risk calculated for each of the two or more second moving objects when the two or more second moving objects have a predetermined correlation. For example, the risk correction unit 25 may use a table including the type, attribute, and correction value of the second moving object (for example, referring to Fig.10 The risk correction unit 25 can realize the above correction function by executing a predetermined program stored in the memory by the processor. The risk correction unit 25 is an example of an execution unit.

[0125] Specifically, when the detected two or more second moving bodies are, for example, two second moving bodies consisting of a person over a predetermined age and a child under a predetermined age, the risk correction unit 25 corrects the individual risk as follows according to the table. That is, the risk correction unit 25 may correct the individual risk calculated for the second moving body of the child among the two second moving bodies in such a way as to reduce the individual risk calculated, without correcting the individual risk calculated for the second moving body of the person over a predetermined age among the two second moving bodies.

[0126] Furthermore, when each of the detected two or more second moving objects is, for example, a child younger than a predetermined age, the risk correction unit 25 may perform correction based on the table so as to increase the individual risk calculated for each of the two or more second moving objects.

[0127] Furthermore, the risk correction unit 25 may correct the individual risk as follows according to the table when the detected two or more second moving bodies are composed of, for example, an adult and two or more children younger than a specified age. That is, the risk correction unit 25 may also correct the individual risk calculated for each of the two or more second moving bodies of the two or more children in the two or more second moving bodies by reducing the individual risk, without correcting the individual risk calculated for the second moving body of the adult. In addition, the degree of reduction of the individual risk calculated for each of the two or more children in the group composed of an adult and two or more children may be made smaller than the degree of reduction of the individual risk calculated for the children in the group composed of an adult and one child. In the case where one adult is caring for two children, it is considered that the risk is lower than that of the child alone. On the other hand, it is considered that it is more difficult for one adult to care for two children than for one adult to care for one child, because the risk of one child in the case of one adult caring for two children is higher than the risk of one child in the case of one adult caring for one child.

[0128] Fig.10An example of the risk correction value according to the present embodiment is shown.

[0129] Fig.10 Table T5 shown in FIG. 5 shows an example of risk correction values ​​corresponding to the type and attribute of the second moving object. Fig.10 In the embodiment, the risk correction value is not limited to being determined according to the type and attribute of the second mobile body, and may be determined according to the positional relationship between the first mobile body M1 and each of the two or more second mobile bodies. In this case, a table showing the risk correction value corresponding to the positional relationship is generated in advance.

[0130] In this embodiment, when the group relationship determination unit 24 determines that the plurality of second mobile bodies belong to a group, the risk correction unit 25 performs individual risk correction (risk correction) by taking into account the influence of other second mobile bodies in the group to which they belong. Fig.10 The individual risk calculated by the individual risk calculation unit 23 is corrected by using Table T5.

[0131] Here, for example, the plurality of second mobile bodies belong to a group consisting of a parent and a child, the parent and a child consisting of a 30-year-old pedestrian and a 5-year-old pedestrian, wherein the 30-year-old pedestrian neither holds the hand of the 5-year-old pedestrian nor hugs the 5-year-old pedestrian. In this case, the individual risk of the 30-year-old pedestrian, for example, based on Figure 5A Table T1 calculates the risk value of pedestrians over 9 years old as [5], and no risk correction is performed. In addition, the individual risk of pedestrians aged 5 years old is calculated based on Figure 5A The risk value of pedestrians under 9 years old is calculated from Table T1 as

[15] , based on Fig.10 Table T5 shown calculates the risk correction value for pedestrians over 9 years old (monitoring) as [-5], so the corrected individual risk is calculated as

[10] .

[0132] Next, use Figures 11 to 14 , individual risks corrected when a plurality of second mobile bodies are in a situation having a predetermined correlation are described.

[0133] Figures 11 to 14 An example of risk correction based on the influence from other second moving objects in the group according to the present embodiment is conceptually shown.

[0134] Fig.11The example shown is a case where risk correction is not performed in a situation where the second moving body A1 and the second moving body A2, both adult pedestrians, belong to a group. It is considered that the individual risk of each adult pedestrian will not change due to the influence of the adult pedestrian. Therefore, in a situation where the second moving body A1 and the second moving body A2, both adult pedestrians, belong to a group having a predetermined correlation, the risk correction unit 25 does not perform risk correction on the calculated individual risk.

[0135] Furthermore, even in the case where both adult pedestrians are adults, the risk correction unit 25 may perform risk correction on the calculated individual risk based on the attributes or status of the adult pedestrians. The attributes here include high school students, college students, etc., and the status includes drunkenness, etc. For example, the risk correction unit 25 may perform correction to increase the individual risk when the adult pedestrians are playing around. The risk correction unit 25 may also perform the correction by establishing a corresponding table between the attributes or status of the adult pedestrians and the risk correction value. Furthermore, the risk correction unit 25 may perform correction to increase the individual risk of one of the adult pedestrians, or may perform correction to increase the individual risk of both of them. High school students, college students, drunkenness, etc., for example, can be obtained through image analysis of images.

[0136] Fig.12 The example shown is a case where the second moving body A1 which is an adult pedestrian and the second moving body C1 which is a child pedestrian are in a group and only the individual risk of the second moving body C1 is corrected. Fig.12 In (a), the calculated individual risks of the second moving body A1 and the second moving body C1 are calculated based on the area R A 1 and region R C 1b. Area R A 1 and region R C 1b indicates, for example, a range in which the first mobile body M1 should sound an alarm in order to avoid danger, and a larger area indicates a greater risk. Fig.12 (b) shows that the individual risk region R of the second moving object C1 is reduced by performing risk correction. C 1b is corrected to region R C 1a. The reason for this is that it can be considered that in a group consisting of an adult and a child, the risk is lower than if the child is calculated alone, because the adult will be watching over the child and can restrain the child from taking dangerous actions.

[0137] Fig.13The example shown is a case where the second moving body A1, which is an adult pedestrian, and the second moving bodies C2, C3, which are child pedestrians, are in a group and only the individual risks of the second moving bodies C2, C3 are corrected. Fig.13 In (a), the calculated individual risks of the second moving body A1, the second moving body C2, and the second moving body C3 are calculated based on the area R A 1. Region R C 2b and region R C 3b size is used to indicate this. Fig.13 (b) shows that the individual risk region R is reduced by performing risk correction to reduce the individual risk of the second moving object C2 and the second moving object C3. C 2b and region R C 3b is corrected to region R C 2a and region R C 3a. The reason is that in a group consisting of one adult and two children, the adult will take care of the children and can suppress the children's dangerous behavior, so the risk is lower than the risk of calculating the children alone. On the other hand, it is considered that the suppression of the children's dangerous behavior is not enough to compare with the case where one adult takes care of one child. Therefore, although the risk of each of the two children becomes smaller than the case where the risk of each child is calculated individually, the risk of each of the two children will still be greater than the case where one adult takes care of one child.

[0138] Fig.14 The example shown is a case where the individual risks of the second moving object C4 and the second moving object C5, both of which are pedestrians who are children, are corrected when they belong to a group. Fig.14 In (a), the calculated individual risks of the second moving object C4 and the second moving object C5 are also based on the area R C 4b and region R C 5b in size. Fig.14 (b) shows that the individual risk region R is increased by performing risk correction to increase the individual risk of the second moving object C4 and the second moving object C5. C 4b and region R C 5b is corrected to region R C 4a and R C 5a. This is because, in a group consisting of two children, it is considered that the possibility of the children's dangerous actions being superimposed is high because no adults are around, so the risk of each of the two children will be greater than when the risks of the two children are calculated separately.

[0139] Furthermore, even when the second moving body C4 and the second moving body C5, both of which are children, are in a group, the individual risks of the second moving body C4 and the second moving body C5 may not be corrected. Figure 7 Each of the judgment criteria shown sets whether to correct individual risks when the pedestrians are all children. Alternatively, for example, when the pedestrians are all children, whether to correct individual risks may be set in advance, and the risk correction unit 25 may uniformly perform control after the setting.

[0140] [1.3.7 Information generation unit 26]

[0141] The information generating unit 26 generates at least one of control information for controlling the first mobile body M1 and notification information for notifying a remote monitor who remotely monitors or operates the first mobile body M1, based on the individual risk corrected by the risk correcting unit 25. The information generating unit 26 can realize the generating function by executing a predetermined program stored in a memory by a processor.

[0142] For example, the information generating unit 26 may generate, as control information, information for causing the first moving body M1 to perform at least one of the following controls, namely, a temporary stop control, an avoidance action control, a control by voice for drawing attention, and a lighting control, according to the corrected individual risk value. And the information generating unit 26 may generate, as control information, information for causing the first moving body M1 to restart driving. For example, the information generating unit 26 may generate, as notification information, alarm information indicating an alarm that prompts the remote monitor to watch or operate the first moving body M1, according to the corrected individual risk value. Here, the notification information may also include information indicating at least one of the size of the alarm display, the color of the alarm display, the presence or absence of flashing of the alarm display, the size of the notification sound for notifying the alarm, and the height of the notification sound.

[0143] In this embodiment, the information generating unit 26 determines the control contents for the terminal D1 and the first mobile body M1 based on the individual risks corrected by the risk correcting unit 25. The control contents include a case where only the control information for the first mobile body M1 is included, a case where only the notification information for the terminal D1 is included, and a case where both the control information and the notification information for the terminal D1 are included.

[0144] Fig.15 An example of information generated according to the corrected individual risk value according to the present embodiment is shown. Fig.15 Table T6 shown is an example of information in which risks and control contents are associated with each other.

[0145] exist Fig.15 Table T6 of FIG. 1 shows an example of a combination of control information and notification information generated according to the corrected individual risk value and the content thereof, as information that associates risk with control content. Fig.15 As shown, when the corrected individual risk value is, for example, 0 to 9, the information generation unit 26 does not generate control information and notification information according to Table T6. Furthermore, when the corrected individual risk value is 10 to 19, the information generation unit 26 generates control information for decelerating the first mobile body M1 and generates notification information for the terminal D1 without sound. Furthermore, when the corrected individual risk value is 20 to 29, the information generation unit 26 only generates notification information including a signal for instructing the terminal D1 to display the alarm A. When the corrected individual risk value is 30 or more, the information generation unit 26 generates both the control information for the first mobile body M1 and the notification information for the terminal D1, but a detailed description is omitted here.

[0146] Furthermore, the notification information to the terminal D1 may include at least one of instruction information for instructing content drawing, alarm display such as blinking an object, enabling or disabling a button, and sound output.

[0147] Furthermore, the control information of the first mobile body M1 may include at least one of information for causing the first mobile body M1 to perform speed control, steering control, image display control, sound output control, etc. The control information may be further generated based on information other than the corrected individual risk. For example, in a case where information is obtained that a worker operating the first mobile body M1 is approaching, even if the corrected individual risk is high, the information generating unit 26 may not include instruction information for instructing the terminal D1 to display an alarm, etc. in the notification information, but only generate the control information of the first mobile body M1.

[0148] Furthermore, the information generation unit 26 may generate instruction information for instructing the terminal D1 to display an alarm, etc., taking into account not only the individual risk corrected by the risk correction unit 25 but also the status of the group to which the two or more second moving bodies belong. For example, in order to notify the remote monitor of the status of the group to which the two or more second moving bodies belong, the information generation unit 26 may generate instruction information for instructing to display an alarm consisting of sentences such as "a group of pedestrians is approaching", "parents and children holding hands are approaching", etc.

[0149] In particular, when the group consists of adults and young children, depending on the timing at which the remote monitor looks at the screen, it may be difficult to tell from the screen that it is a group if the remote monitor does not carefully check whether there are children behind the adults. By notifying the alarm, the remote monitor can understand how many people are in the group, and can avoid missing the young children in the above example.

[0150] Alternatively, the alarm display may include a display that can be identified as the same group. For example, the alarm display may include a display that emphasizes the group to which two or more second mobile bodies belong. For example, the alarm display may include a display that surrounds all members of the same group in a rectangle, or the display mode of the frames representing the members of the same group may be set to be the same. Also, it may be that, considering that the degree of individual risk is different according to the different reasons for judging that they belong to the same group, the display mode of the frame is different according to the reason for judging that they belong to the same group or the degree of risk corresponding to the reason for judging. The display mode of the frame includes at least one of the form of the frame line and the form of the color representing the group. Also, it may be that the alarm display is a display that simply overlaps the reason for judgment and the degree of risk. For example, in particular, when it is judged that the group is physically connected, such as a situation where a young child is held by the hand of a parent or is held, the risk of the child approaching will be reduced, but when it is judged that the group is not physically connected but is close, the risk of the child approaching will not be very small. Compared with the former, the latter case requires the remote monitor to pay attention. Therefore, the alarm display may be displayed to emphasize the latter case more than the former.

[0151] Furthermore, the alarm display may be performed when the situation of the group changes. For example, when the distance changes from a state of physical connection to a state of no physical connection and the distance becomes closer, an alarm display is performed in this case because more attention is needed, and the alarm display is effective. Furthermore, in the opposite case, no alarm display is performed, or by reducing the emphasis of the alarm display, the workload of the remote monitor when performing attention work can be reduced, and the remote monitor can increase the attention work to other places. The alarm display may also be performed in a manner that can identify whether the alarm display is an alarm that requires more attention or an alarm that can reduce the workload of attention work. As such, for example, changing the display color can be cited, but it is not limited to this. Making the alarm display method different is an example of performing a prescribed work.

[0152] Furthermore, the alarm display may include a display showing the attributes of the group, and may also include a display showing the number of second moving objects included in the group. The display showing the attributes of the group may include, for example, parents and children, adults and adults, children and children, people in wheelchairs and people pushing wheelchairs, people and pets, etc., but is not limited thereto.

[0153] Furthermore, when the image is a momentary image with a child behind an adult, even a computer cannot easily determine whether it is a group. If the computer determines that it is a group before a predetermined period, the information generating unit 26 will infer that it is a group because there is a child behind the adult, even if only one adult is shown in the current image, and will display an alarm to indicate that it is a group. For example, the information generating unit 26 may include information indicating that a person in the group cannot be confirmed on the screen (for example, a display such as "child in blind spot") in the alarm display. Furthermore, for example, the information generating unit 26 may include the number of people who cannot be confirmed in the alarm display.

[0154] Furthermore, the information generating unit 26 may generate the control information of the first moving body M1 by giving priority to the situation of the group to which the two or more second moving bodies belong over the individual risk corrected by the risk correcting unit 25. For example, when a teacher and a plurality of children led by the teacher are in a situation of belonging to a group, even if it is determined based on the risk value that an alarm needs to be issued and the first moving body M1 needs to be controlled, the information generating unit 26 may not generate the control information of the first moving body M1 even if the corrected individual risk is high but below the threshold value because the group is safe due to being led by the teacher.

[0155] [1.3.8 Output section 27]

[0156] The output unit 27 outputs at least one of the control information and the notification information generated by the information generating unit 26 .

[0157] In this embodiment, the output unit 27 transmits the control content determined by the information generating unit 26, i.e., at least one of the control information and the notification information generated by the information generating unit 26, to the terminal D1 via the communication unit 11. The output unit 27 can also transmit the control content determined by the information generating unit 26 to the first mobile body M1.

[0158] [2 Work]

[0159] The following describes an information processing method executed by the information processing device 10 having the above-described configuration, that is, an operation of the information processing device 10 .

[0160] [2.1 Overall composition work]

[0161] First, use Fig.16 The information transmission and reception etc. between the information processing device 10, the first mobile object M1, and the terminal D1 will be described.

[0162] Fig.16 It is a sequence diagram showing the operation (information processing method) including the overall configuration of the information processing device 10 according to the present embodiment. Fig.16 The diagram shows an example of a sequence diagram in which the first mobile body M1 transmits state information and sensing information to the information processing device 10 , the information processing device 10 determines control content, and the terminal D1 performs control according to the control content.

[0163] First, the first moving body M1 obtains state information through the state information obtaining device M13 ( S111 ), and transmits the obtained state information to the information processing device 10 through the network N through the communication device M11 ( S112 ).

[0164] Next, the first mobile body M1 obtains sensing information through the sensing device M12 (S113), and transmits the obtained sensing information to the information processing device 10 through the communication device M11 via the network N (S114). In addition, the timing of sending the state information and the timing of sending the sensing information may be the same or different. Furthermore, the frequency of obtaining and sending the state information and the frequency of obtaining and sending the sensing information may be different.

[0165] Next, when the information processing device 10 receives the state information and the sensed information via the network N ( S121 ), it performs information processing using the received state information and the sensed information and determines the control content ( S122 ). The details of the information processing will be described later.

[0166] Next, the information processing device 10 transmits the determined control content to the terminal D1 via the network N ( S123 ).

[0167] Next, the terminal D1 receives the control content through the communication device D11 (S131), and performs control according to the received control content (S132).

[0168] In addition, depending on the control content determined in step S122, Fig.16In addition to the case where only the terminal D1 performs control according to the control content in the example shown, there is also a case where the control content is sent only to the first mobile body M1 and the control is performed only by the first mobile body M1. In addition, it is also possible to consider a case where the control content is sent to both the terminal D1 and the first mobile body M1 at the same time and the control is performed by the terminal D1 and the first mobile body M1. In addition, it is also possible to consider a case where the control content is not sent to the terminal D1 and the first mobile body M1 and the control is not performed by the terminal D1 and the first mobile body M1.

[0169] In addition, although Fig.16 , there is one terminal D1 and one first moving body M1, but there may be multiple terminals. In this case, the remote monitor can control a specific terminal D1 among the multiple terminals D1, but when there are multiple remote monitors, all terminals D1 can be controlled.

[0170] [2.2 Operation of Information Processing Device 10]

[0171] Next, use Fig.17 as well as Fig.18 Yes Fig.16 An example of the operation of the information processing device 10 performed in steps S121 to S123 will be described.

[0172] Fig.17 This is a flowchart showing an example of the operation (information processing method) of the information processing device 10 according to the present embodiment. Fig.17 An example of processing performed by the processing unit 20 of the information processing device 10 according to the present embodiment is shown.

[0173] like Fig.17 As shown, first, the information processing device 10 obtains state information and sensing information (S301). In this embodiment, the information processing device 10 receives the image information obtained by the sensing device M12 such as the vehicle-mounted camera of the first moving body M1, i.e., sensing information, and the state information obtained by the state information obtaining device M13 through the communication unit 11 via the network N. Alternatively, the state information may include position information, speed information, tilt information, orientation information, and information on display content of the first moving body M1.

[0174] Next, the information processing device 10 detects the second moving object based on the sensing information obtained in step S301 (S302). In this embodiment, the second moving object detection unit 21 of the information processing device 10 performs image recognition by using the obtained image information and the information required for the detection of the second moving object, thereby detecting two or more second moving objects. In addition, each of the two or more second moving objects can be a pedestrian, a person riding on a bicycle, a wheelchair, a baby carriage, an electric wheelchair for the elderly, or a pet, etc.

[0175] Next, the information processing device 10 determines the position information of each second mobile body based on the state information and sensing information obtained in step S301 (S303). In the present embodiment, the second mobile body position determination unit 22 of the information processing device 10 uses the obtained state information and the detected position information of the second mobile body on the image to calculate the position coordinates of the second mobile body relative to the first mobile body M1. Alternatively, when the state information includes the position coordinates, direction, camera position information, camera tilt, and camera viewing angle of the first mobile body M1, the second mobile body position determination unit 22 uses the information included in the state information to calculate the position coordinates of the second mobile body. Furthermore, the second mobile body position determination unit 22 further calculates the speed information and acceleration information of the second mobile body relative to the first mobile body M1 based on the calculated time change of the position information of the second mobile body.

[0176] Next, the information processing device 10 calculates the individual risk of each second moving body based on the state information obtained in step S301 and the position coordinates determined in step S303 (S304). In this embodiment, the individual risk calculation unit 23 of the information processing device 10 calculates the individual risk of each second moving body. The individual risk calculation unit 23 may also calculate the individual risk of each second moving body by comparing the respective attributes of the second moving body determined in step S302, the position information, speed information, acceleration information, etc. obtained in step S303, and the reference information when calculating the risk.

[0177] Here, the use of Figure 5A Table T1 to Figure 5D The following describes an example of calculating individual risks using Table T4. For example, assume that the individual risks of two second moving objects, an adult pedestrian over 9 years old and a child pedestrian under 9 years old, are calculated. In this case, the individual risk calculation unit 23 refers to Figure 5ATable T1 can calculate the risk value of pedestrians over 9 years old as [3], and the risk value of pedestrians under 9 years old as

[15] . In addition, when the distance between the two second moving bodies and the first moving body M1 as a vehicle is 3m, the individual risk calculation unit 23 refers to Figure 5B Table T2 can calculate the risk values ​​of the two second moving bodies as [5]. In addition, when the moving speed of the first moving body M1 included in the state information is 3 km / h, the individual risk calculation unit 23 refers to Figure 5C The risk values ​​of the two second moving bodies can be calculated as [3]. In addition, the individual risk calculation unit 23 refers to the table T3, for example, when the speed of the two second moving bodies is 7 km / h. Figure 5D Table T4 can calculate the risk values ​​of the two second moving bodies as [8]. In this way, the individual risk calculation unit 23 combines these risk values ​​and calculates them as individual risks. Fig.17 Continue with the description.

[0178] Next, the information processing device 10 determines whether two or more second mobile bodies are in a state of having a prescribed association (S305). In the present embodiment, when the information processing device 10 detects that there are multiple second mobile bodies, the group relationship determination unit 24 determines whether the multiple second mobile bodies have a prescribed relationship as belonging to a group. For example, the group relationship determination unit 24 may determine the group of multiple second mobile bodies and the persons who make them up. In this way, the group relationship determination unit 24 may also determine whether the multiple second mobile bodies belong to a group based on the group determination information maintained in the storage unit 12, the attributes of the determined second mobile body, the calculated position information of the second mobile body relative to the first mobile body M1, and the like.

[0179] Here, a specific example in which two or more second moving bodies are judged to have a predetermined association is described. For example, the two or more second moving bodies are an adult and one or more children. In this way, it is assumed that it can be determined based on the position information determined in step S303 that the adult and the child are in a situation where they stand for a certain time or more within a certain distance or move in the same direction at the same speed within a certain distance. In this case, the group relationship determination unit 24 can determine that the two or more second moving bodies belong to a group with a predetermined association. In addition, it is assumed that a situation in which an adult holds a child's hand, an adult holds a child, a person leads a pet, and other situations in which there are physical constraints on each other can be determined based on image information. In this case, the group relationship determination unit 24 can determine that the two or more second moving bodies have a predetermined association and belong to a group. In addition, it is assumed that a situation in which an adult takes care of a child, a situation in which people look at each other, a situation in which a person who looks at each other exists for a certain time or more, and other situations in which there are people who look at each other can be determined based on image information. In this case, the group relationship determination unit 24 can determine that the two or more second moving bodies consisting of a person who looks at and a person who is looked at have a predetermined association and belong to a group. Furthermore, it is assumed that a situation in which a conversation is in progress, such as a situation in which an adult calls a child or a parent and a child are having a conversation, can be determined based on the sound information around the first mobile object M1 included in the state information. In this case, the group relationship determination unit 24 can determine that the two or more second mobile objects in conversation have a predetermined relationship and belong to a group.

[0180] In step S305 , when the plurality of second moving objects that do not have the predetermined association are determined not to belong to the group (No in S305 ), the information processing device 10 proceeds to step S307 .

[0181] In addition, in step S305, when a plurality of second mobile bodies having a predetermined correlation are determined to belong to a group ("Yes" in S305), the information processing device 10 corrects the individual risk calculated in step S304 (S306). For example, the risk correction unit 25 of the information processing device 10 corrects the individual risk of each of the second mobile bodies determined to belong to the group in step S305. That is, the risk correction unit 25 corrects (risk correction) the individual risk of each of the plurality of second mobile bodies calculated by the individual risk calculation unit 23 by taking into account the influence from other second mobile bodies belonging to the group.

[0182] Here, the use of Fig.10 Table T5 is used to illustrate an example of the case of correcting individual risk. For example, two second moving objects, a 30-year-old pedestrian and a 5-year-old child pedestrian, are judged to belong to a group, based on Figure 5ATable T1 shows that the individual risk for a 30-year-old pedestrian is [3], and the individual risk for a 5-year-old child is

[15] . Fig.10 As shown in Table T5, when the conditions of the two second moving objects belonging to the group are the following, namely, when a pedestrian over 9 years old is taking care of the other moving objects, holding the hands of the other moving objects, or hugging the other moving objects, the risk correction value of the individual risk is different.

[0183] For example, in the case where the parent and child belonging to the group are in a situation where the parent is taking care of the child, the risk correction unit 25 refers to Fig.10 Table T5 shows that the risk correction value of the other second moving objects can be [-5] due to the influence of pedestrians over 9 years old. Therefore, the risk correction unit 25 can calculate the corrected individual risk of [+15] for the child by the risk correction value [-5]. In addition, for example, in the case where the parent is holding the child's hand as a parent and the child belongs to the group, the risk correction unit 25 refers to Fig.10 Table T5 shows that the risk correction value of the other second moving objects can be [-12] due to the influence of pedestrians over 9 years old. Therefore, the risk correction unit 25 can calculate the corrected individual risk of [+3] by correcting the individual risk of the child [+15] with the risk correction value [-12]. In addition, for example, in the case where the parent is holding the child as a group of parents, the risk correction unit 25 refers to Fig.10 According to Table T5, the risk correction value of the other second moving objects can be obtained as [-15] due to the influence of pedestrians over 9 years old. Therefore, the risk correction unit 25 can calculate the corrected individual risk as [+0] by correcting the individual risk [+15] of the child with the risk correction value [-15].

[0184] However, in the case where the parent and child are in a group and the parent is looking after the child, there is no risk correction value corresponding to pedestrians under 9 years old for the other second moving objects. Therefore, the risk correction unit 25 does not correct the individual risk [+3] of the parent or corrects the individual risk [+3] of the parent by the risk correction value [9], and can calculate the corrected individual risk as [+3].

[0185] In addition, the conditions of two or more second moving bodies may change over time. For this reason, steps S305 and S306 may be repeated at regular intervals to update the corrected individual risk. For example, when a parent and a child holding hands let go, it may be detected, for example, through image recognition, that the conditions of the group to which the two second moving bodies belong have changed. Accordingly, referring to Fig.10The risk-adjusted values ​​obtained from Table T5 will change, and therefore the values ​​of the corrected individual risks will also change.

[0186] Furthermore, for example, a 30-year-old pedestrian and three 5-year-old children, that is, four second moving objects, are determined to belong to a group. Figure 5A Table T1 calculates the individual risk of a 30-year-old pedestrian as [3], and the individual risk of a 5-year-old child as

[15] . In this case, as the risk correction value for a child, Fig.10 Table T5 shows that the risk correction value of the second moving object affected by the pedestrian over 9 years old is [-5], and the risk correction value of the other moving object for the two pedestrians under 9 years old is [+10 (the value obtained by [5]×[2])]. Therefore, the risk correction unit 25 corrects the individual risk of each child [+15] by the risk correction value [+5 (the value obtained by [-5]+

[10] )], and can calculate the corrected individual risk of each child as [+20]. Next, return to Fig.17 Continue with the description.

[0187] In addition, although the above description has been given of an example in which the risk correction unit 25 does not correct the individual risk of the parent, it is not limited thereto, and the individual risk of the parent may also be corrected. For example, in the case where the parent is caring for the child as a parent and the child belongs to a group, the risk correction value of the other second moving body (here, the parent) affected by the child is included in Table T5, and the risk correction unit 25 corrects the individual risk of the parent based on the Table T5. For example, in the case where the parent is caring for the child as a parent and the child belongs to a group, the risk correction unit 25 performs a correction that increases the individual risk of the parent.

[0188] Next, the information processing device 10 generates at least one of control information for controlling the first mobile body M1 and notification information for notifying a remote monitor who is remotely monitoring or operating the first mobile body M1 based on the individual risk corrected in step S306 (S307). For example, the information generation unit 26 of the information processing device 10 determines the control content for the terminal D1 and the first mobile body M1 based on the individual risk corrected in step S306. In this embodiment, the information generation unit 26 can use, for example, Fig.15 The displayed risk is associated with the control content, and the control content is determined using the uncorrected individual risk or the corrected individual risk.

[0189] Here, the use of Fig.15The following describes an example of determining the control content by using Table T6. For example, among a group of one 30-year-old pedestrian and three 5-year-old child pedestrians, the corrected individual risk for each of the child pedestrians is

[20] . In this case, the information generation unit 26 determines the control content according to Table T6. Fig.15 The control content is determined by using the table T6, and control information for the first moving body M1 is not generated, but notification information for causing the alarm A to be displayed on the terminal D1 is generated.

[0190] Next, the information processing device 10 outputs at least one of the control information and the notification information generated in step S307 ( S308 ). For example, the output unit 27 of the information processing device 10 transmits the control information determined by the information generating unit 26 in step S307 via the communication unit 11 .

[0191] Fig.18 It is shown Fig.17 A flowchart of an example of the details of step S307 is shown.

[0192] In step S307, first, the information processing device 10 determines whether it is necessary to control the first moving body M1 (S3071). For example, the information generation unit 26 of the information processing device 10 refers to the information associated with the risk and the control content stored in the storage unit 12, and can determine whether it is necessary to control the first moving body M1 based on the individual risk corrected in step S306.

[0193] In step S3071 , when it is determined that the first moving body M1 needs to be controlled (Yes in S3071 ), the information generating unit 26 generates control information ( S3072 ).

[0194] In addition, in step S3071, when it is determined that the first moving body M1 does not need to be controlled ("No" in S3071), the information generation unit 26 determines whether the operator (remote monitor) needs to watch or intervene in the first moving body M1 (S3073). For example, the information generation unit 26 of the information processing device 10 refers to the information associated with the risk and the control content stored in the storage unit 12, and can determine whether the operator needs to watch or intervene in the first moving body M1 based on the individual risk corrected in step S306.

[0195] In step S3073 , when it is determined that the operator's attention or intervention is necessary (Yes in S3073 ), the information generating unit 26 generates notification information ( S3074 ).

[0196] In addition, in step S3073, when it is determined that the operator's attention or intervention is not necessary (No in S3073), the processing of step S307 is terminated.

[0197] [3 Effects, etc.]

[0198] As described above, through this embodiment, it is possible to realize an information processing method that can more accurately calculate the risk of mobile bodies such as pedestrians and perform necessary and sufficient control on mobile bodies such as vehicles. More specifically, through the embodiment, by correcting the risk corresponding to the situation in the group for the second mobile body that has a prescribed relationship and belongs to the group, a more accurate risk assessment can be performed. In this way, by performing an alarm notification to the terminal D1 or controlling the first mobile body M1 based on the correct risk assessment, the number of interventions by the remote monitor can be suppressed and the load can be reduced.

[0199] The following describes how individual risks are calculated and corrected in a scenario where a parent and a child are approaching in front of a vehicle by taking a comparative example.

[0200] Fig.19A The risk calculated by the method according to the comparative example in a certain scenario is shown. Fig.19B The risk calculated by the method according to the present disclosure in a certain scenario is shown.

[0201] Fig.19A as well as Fig.19B (a) shows an example of a scene in which a parent and a child are approaching the vehicle in front of the vehicle. Fig.19A as well as Fig.19B (b) shows an example of individual risk calculated for the size of the area for each parent and child. The size of the area is the range where the vehicle should sound the alarm to avoid danger, and the larger the calculated individual risk, the larger the range. In other words, the calculated individual risk shows that the risk for adults as parents is small, while the risk for children is large.

[0202] Fig.19A (c) shows that according to Fig.19A (b) The individual risk calculated by the vehicle alarm is issued. That is, the comparative example does not take into account the relationship between adults and children, that is, it does not take into account that the risk of children being cared for by their parents (adults) should be lower than the risk of children being alone. As a result, unnecessary control of the vehicle may occur, resulting in a decrease in the efficiency of vehicle operation.

[0203] However, in Fig.19B (c) of the paper takes into account the situation within the parent-child group. Fig.19B The individual risk calculated by (b) is corrected. Fig.19B (d) shows that, according to Fig.19B (c) Correction of individual risks, vehicle alarms, etc.

[0204] When Fig.19B (d) and Fig.19A When compared with (c), even if the scenario is the same where the parent and child are approaching the vehicle in front of the vehicle, the calculated individual risk of the child is different. That is, by correcting the risk of the children belonging to the group according to the situation within the group where there is a parent-child relationship and the child is being cared for by the parents, it is possible to obtain a more correctly assessed risk (corrected individual risk). Based on this, it is possible to perform necessary and sufficient control on moving objects such as vehicles.

[0205] (Various Modifications of Embodiments)

[0206] Below, refer to Figure 20 to Figure 22 Various modifications of the embodiment will be described. In addition, the following description will focus on the differences from the embodiment, and the description of the same or similar contents as the embodiment will be omitted or simplified.

[0207] First, refer to Fig. 20 Another example in which individual risks of a plurality of second moving objects in situations having a predetermined correlation are corrected will be described. Fig. 20 An example of risk correction based on the influence from other second moving objects in the group according to this modification is conceptually shown.

[0208] Fig. 20 The example shown is that when the second moving body A1, which is an adult pedestrian, and the second moving body C1, which is a child pedestrian, belong to a group, only the individual risk of the second moving body C1 is corrected. Fig. 20 In (a), the calculated individual risks of the second moving body A1 and the second moving body C1 are calculated based on the area R A 1 and region R C 1b. Area R A 1 and region R C 1b shows, for example, the range in which the first moving body M1 should sound an alarm in order to avoid danger. The larger the range, the greater the risk. Here, the second moving bodies A1 and C1 are adjacent, so area R C 1b includes area R A 1 area. In addition, area R A 1 only needs to include the area between the second moving body A1 and the first moving body M1, and may be an area not including the second moving body A1, for example. Also, the second moving body A1 and the second moving body C1 may be located in the area R, for example. C Position within 1b.

[0209] Fig. 20(b) shows that the individual risk region R of the second moving object C1 is reduced by performing risk correction. C 1b is corrected to region R C 1a. That is, in a group consisting of one adult and two children, the adult will be watching over the children and will be able to restrain the children from taking dangerous actions, so the individual risk will be smaller than when the risk of the children is calculated separately. C 1a is with area R A 1 is equal to the area. In addition, the area R C 1a can also be larger than area R C 1b is small and includes area R A 1 area.

[0210] Thus, in the area R corresponding to the child pedestrian C 1b includes all regions R corresponding to adult pedestrians A 1, the corrected area R corresponding to the child pedestrian C 1a can also be corrected to the area R corresponding to an adult pedestrian A 1. In other words, even if the first moving body M1 enters the area R C 1b, the alarm will not be activated, and the first moving body M1 can enter the area R A 1, the alarm is triggered to correct the area R C 1b.

[0211] Next, an example is described in which, when determining whether a plurality of second moving bodies have a predetermined correlation (group determination), the risk level (risk correction value) is corrected differently according to the determination criteria satisfied. Figure 7 If the judgment is "yes" in any of the judgments shown, Fig.10 The risk-adjusted values ​​shown are different. For example, Figure 7 Set the judgment criteria shown in Fig.10 The table T5 shown in FIG. 1 can also be used by the risk correction unit 25 to adjust the risk according to the judgment criteria met. Fig.10 The risk-corrected values ​​in Table T5 are shown.

[0212] For example, when there is a physical connection (such as holding hands, hugging, etc.), since safety is high, the risk correction value is further corrected in the negative direction to become a larger value, so that the corrected individual risk becomes a smaller value. And for example, when the distance between the two second moving bodies is closer than a specified distance, since safety is high, the risk correction value is further corrected in the negative direction to become a larger value, so that the corrected individual risk becomes a smaller value. The correction amount of the risk correction value may also be set according to safety. For example, since safety is higher when there is a physical connection compared to when the distance between the two second moving bodies is closer than a specified distance, the correction amount when there is a physical connection is set to be larger. For example, a table that establishes a correspondence between the correction amount of the risk correction value and safety may be stored in the storage unit 12. In this case, according to Figure 7 Set security based on the various judgment criteria shown.

[0213] Next, refer to Fig.21 An example of making the voices emitted from the first mobile body M1 to two or more second mobile bodies belonging to a group different according to the association of the group determined when group determination is performed is described below. An example of making the voices emitted from the first mobile body M1 to a group different according to the association of the group is described below. In addition, the work for making the voices different is an example of a prescribed work.

[0214] For example, when the group's association is that of an adult and a child, the first mobile body M1 issues an alarm to urge the adult to hold the child's hand, and when the group's association is that of an adult and an adult, the first mobile body M1 issues a normal alarm (for example, a sound of "a vehicle is about to pass"). A table that establishes a correspondence between associations and alarm methods is stored in the storage unit 12, and the information generation unit 26 may also generate control information including the alarm method.

[0215] Fig.21 This is a diagram for explaining control of a first moving object when two second moving objects having a predetermined correlation according to the present modification avoid an obstacle. Fig.21 (a) shows that the first moving body M1 and two second moving bodies A1 and C1 having a predetermined association are moving toward each other, and there is an obstacle between them. Fig.21 (b) shows that the first moving body M1 stops to allow the two second moving bodies A1 and C1 to pass through first without obstacles. Fig.21 , a situation where two second moving bodies A1 and C1 walk in a row (an example of movement) is illustrated.

[0216] like Fig.21 As shown in (b), when one of the two second moving bodies A1 and C1 avoids an obstacle, the other second moving body also avoids the obstacle, that is, there is a possibility of avoiding the obstacle while maintaining a horizontal row. In this case, the two second moving bodies A1 and C1 will approach (for example, run out) the side of the first moving body M1.

[0217] In this case, the two second moving bodies A1 and C1 having a predetermined correlation may be regarded as one second moving body, and the collision between the one second moving body and the obstacle may be predicted according to their respective positions, and the first moving body M1 may be controlled according to the prediction result. The vehicle control here includes control to stop the first moving body M1. In addition, the vehicle control may also include control to make the first moving body M1 move slowly or make a detour.

[0218] For example, the information generating unit 26 may generate control information including stopping the first moving body M1 in front of the obstacle when predicting that the one second moving body will collide with the obstacle. Such processing is performed on two or more second moving bodies having a predetermined correlation.

[0219] In addition, Fig.21 In the example, although the case where two second moving bodies with a predetermined association move in a row horizontally is cited, for example, control to stop the first moving body M1 may be performed when the two second moving bodies move in a row vertically. The operation of stopping the first moving body M1 until the two second moving bodies with a predetermined association bypass the obstacle is an example of the predetermined operation.

[0220] Alternatively, the present disclosure may be implemented by the following contents. That is, processing other than the processing shown below may not be performed. Also, the present disclosure may be implemented by an information processing device that executes the following information processing method or as a program for causing a computer to execute the following information processing method.

[0221] An information processing method is an information processing method in an information processing device.

[0222] In the information processing method,

[0223] acquiring sensing information, wherein the sensing information is information obtained by sensing the surroundings of a first moving body that is autonomously moving,

[0224] detecting a second moving object existing around the first moving object based on the sensing information,

[0225] When two or more second moving bodies are detected to be present around the first moving body, determining whether the two or more second moving bodies are in a state having a predetermined correlation based on the sensing information,

[0226] When it is determined that two or more of the second moving objects are in a situation having the predetermined correlation, notification corresponding to the predetermined correlation is performed.

[0227] Next, the display of support information (for example, alarm information) for remote monitoring control and watching will be described.

[0228] Although the above description is about an example in which the information generating unit 26 generates indication information for indicating an alarm display consisting of sentences such as "a group of pedestrians are approaching", "parents holding hands are approaching", etc., the information processing method of the present disclosure can also be implemented as a method in which, without correcting the individual risks of two or more second moving bodies having a prescribed correlation, indication information for indicating an alarm display corresponding to the prescribed correlation is generated for two or more second moving bodies having a prescribed correlation. Furthermore, the work of generating such indication information is an example of prescribed work.

[0229] Next, an example of the age estimation method described above will be described. In addition, the age estimation method is not limited to the method described below, and any known method may be used.

[0230] Age estimation is performed based on image recognition. As image recognition, a processor executes a predetermined program stored in a memory, and for example, image recognition using machine learning based on deep learning or other AI technologies can be cited.

[0231] As a first method, for example, there is a method of estimating age by face recognition. As long as there is no mask, hat, sunglasses, etc., the age can be correctly estimated to some extent. As a second method, for example, there is a method of estimating by height after character recognition. In the case where age estimation needs to be performed with high precision, the first method can be used, and in the case where it is possible to distinguish between children and adults, the second method can be used. In the case of the second method, the information generation unit 26 may also regard the case below the first specified height as a child, and the case above the second specified height as an adult. The first specified height and the second specified height may be the same value or different values. For example, the first specified height may be 120cm and the second specified height may be 155cm. In this case, if the height is greater than 120cm and less than 155cm, the information generation unit 26 may uniformly regard it as an adult, or estimate the age by face recognition, or estimate the age by clothing.

[0232] Next, refer to Fig. 22 Another example of correcting individual risk will be described. Hereinafter, an example in which the risk correction unit 25 further corrects the individual risk when a predetermined condition is satisfied will be described.

[0233] The predetermined condition may also be that there is a person in the group who is listed in a blacklist. The blacklist includes information for identifying a person who has caused an obstruction to the travel of the first moving body M1 in the past, for example, information for identifying a person who has played a prank on the first moving body M1 in the past. The risk correction unit 25 may also correct the group by, for example, further increasing the individual risk when there is a person listed in the blacklist in the group. Fig.10 The risk correction value in Table T5 shown in the figure is used to correct the individual risk. In addition, whether to record a person in the blacklist can be determined by, for example, face recognition, but the present invention is not limited to this.

[0234] Furthermore, although the prescribed conditions are the same conditions within the group (for example, the same position, the same situation, etc.), a second moving body with different individual risks may also exist. The same position includes a position within a prescribed distance. Examples of second moving bodies with the same conditions but different individual risks include parents and children. In this case, the risk correction unit 25 may also make the degree of correction of individual risk different according to whether the second moving body close to the first moving body M1 is a person with high individual risk or a person with low individual risk. For example, the risk correction unit 25 may make a correction to reduce the degree of correction of individual risk (that is, further increase the individual risk) when the party close to the first moving body M1 is a person with high individual risk (for example, a child), and make a correction to increase the degree of correction of individual risk (that is, further reduce the individual risk) when the party close to the first moving body M1 is a person with low individual risk (for example, an adult). The risk correction unit 25 may also make a correction according to which party is on the side of the first moving body M1 when the second moving body with the same conditions but different individual risks exists. Fig.10 The risk-adjusted values ​​in Table T5 are shown, and the adjusted risk-adjusted values ​​are used to correct individual risks.

[0235] Fig. 22 This is a diagram for explaining an example of risk correction based on the positional relationship between the second moving object and the first moving object M1 in the group according to the present modification. Fig. 22 (a) shows the situation where the child among the parents is on the side of the first moving body M1, Fig. 22 (b) shows a case where the parent of the parent and the child is on the side of the first moving object M1. It is assumed that the individual risk of the parent (second moving object A1) is lower than that of the child (second moving object C1).

[0236] exist Fig. 22 In the case shown in (a), since the child with a high individual risk among the parent and the child is on the first moving body M1 side (the child is located close to the first moving body M1), the risk correction unit 25 reduces the degree of correction of the individual risk. In other words, the risk correction unit 25 performs correction in such a way that the individual risk of the child becomes a higher value.

[0237] And, in Fig. 22 In the case shown in (b), since the parent with a low individual risk among the parents and the child is on the first moving body M1 side (the parent is located close to the first moving body M1), the risk correction unit 25 increases the degree of correction of the individual risk. In other words, the risk correction unit 25 performs correction in such a way that the individual risk of the child becomes a lower value.

[0238] In the above-mentioned embodiment, although the individual risk value is calculated and the individual risk calculated for each of the two or more second moving bodies is corrected when the two or more second moving bodies are in a state with a prescribed correlation, the individual risk calculation and the individual risk correction may not be performed. For example, the information processing method in the information processing device involved in one embodiment of the present disclosure only needs to include the following processing, that is, obtaining sensing information obtained by photographing the surroundings of the first moving body M1 that moves autonomously, detecting the second moving body existing around the first moving body M1 based on the sensing information, and when it is detected that there are two or more second moving bodies existing around the first moving body M1, judging whether the two or more second moving bodies are in a state with a prescribed correlation based on the sensing information, and when it is judged that the two or more second moving bodies are in a state with a prescribed correlation, performing a prescribed operation to prevent the two or more second moving bodies from contacting the first moving body M1. The prescribed operation may be an operation for making the display mode of the alarm displayed on the terminal D1 different, or an operation for making the voice to the group emitted from the first mobile body M1 different, or an operation for stopping the first mobile body M1 until two second mobile bodies having a prescribed association bypass the obstacle. Furthermore, the prescribed operation may be an operation for calculating individual risks indicating the possibility of danger of contact between two or more second mobile bodies and the first mobile body M1, and correcting the individual risks calculated for each of the two or more second mobile bodies, or an operation for further notifying the first mobile body M1 and at least one of the remote monitor's terminal of information related to the corrected individual risks. In this case, the information related to the corrected individual risks includes, for example, at least one of control information and notification information.

[0239] (Remark)

[0240] The group in the present disclosure refers to a collection of second moving bodies that have a prescribed association. For example, situations where an adult is looking at a child, a pedestrian is walking ahead with a dog, and children are riding bicycles in the same direction can be judged as having a prescribed association with each other and belonging to a group. In addition, situations where the second moving bodies are staggered, the distance between the second moving bodies is large, and an adult and a child are in a close position but their sight lines do not intersect, etc., are judged as second moving bodies that do not have a prescribed association with each other and are considered not to belong to a group.

[0241] Furthermore, as described above, the risk in the present disclosure shows the probability of a second moving body colliding with a first moving body M1 such as a vehicle. Such a risk is represented, for example, by a numerical value, and the greater the risk value, the higher the danger. The magnitude of the individual risk can be calculated by taking into account physical conditions such as the distance between the first moving body M1 as a vehicle and the second moving body such as a pedestrian, the speed and direction of travel of the first moving body M1, the speed and direction of travel of the second moving body, and the presence or absence of obstacles between the first moving body M1 and the second moving body. Furthermore, in addition to the above-mentioned physical conditions, the magnitude of the individual risk can also be calculated by taking into account the attributes of the second moving body such as the age, gender, and clothing of the second moving body. For example, even under exactly the same physical conditions, the individual risk calculated for the case where the second moving body is an adult pedestrian is greater than that for the case where the second moving body is a child riding a bicycle.

[0242] Based on the description of the above embodiments, the following technology is disclosed.

[0243] (Technology 1) An information processing method is an information processing method in an information processing device, in which sensing information is obtained, the sensing information is obtained by sensing the surroundings of a first moving body that is moving autonomously, and a second moving body that exists around the first moving body is detected based on the sensing information. When it is detected that there are two or more second moving bodies that exist around the first moving body, it is determined based on the sensing information whether the two or more second moving bodies are in a state with a specified correlation. When the two or more second moving bodies are in a state with the specified correlation, a specified operation is performed to avoid contact between the two or more second moving bodies and the first moving body.

[0244] In this way, it is possible to determine whether two or more second moving bodies have a predetermined correlation based on the sensing information, etc., and perform a predetermined operation to prevent the two or more second moving bodies from contacting the first moving body. By performing the predetermined operation in a manner that does not perform unnecessary operations on the first moving body, an information processing method that can perform necessary and sufficient control on the first moving body can be realized.

[0245] (Technique 2) An information processing method as described in Technique 1, wherein an individual risk is calculated based on the first position of the first moving body and the second positions of each of the two or more second moving bodies, wherein the individual risk indicates the possibility of danger of each of the two or more second moving bodies coming into contact with the first moving body, and when the two or more second moving bodies are in a state having the prescribed association, the following work is performed as the prescribed work: the individual risk calculated for each of the two or more second moving bodies is corrected, and information related to the corrected individual risk is notified to at least one of the first moving body and a terminal of a remote monitor, wherein the remote monitor is a monitor who monitors or operates the first moving body remotely.

[0246] According to this, the individual risk calculated for each of the two or more second moving bodies can be corrected. Therefore, since a more accurate risk assessment can be obtained for the second moving body, necessary and sufficient control can be performed on the first moving body. In addition, for example, the number of interventions or remote operations of the remote monitor on the first moving body M1 can be suppressed, thereby reducing the load on the remote monitor.

[0247] (Technique 3) The information processing method as described in Technique 2 obtains status information related to the moving status of the first mobile body, and when two or more of the second mobile bodies are in a state having the specified correlation, determines the second position and attribute of each of the two or more second mobile bodies detected, calculates the individual risk of each of the two or more second mobile bodies based on the status information and the determined second position of each of the two or more second mobile bodies, and determines whether the two or more second mobile bodies are in a state having the specified correlation based on the determined second position and attribute of each of the two or more second mobile bodies.

[0248] According to this, since the state information and the attributes can be used, it is possible to more accurately calculate the individual risk and determine whether or not two or more second mobile objects are in a situation having a predetermined correlation.

[0249] (Technique 4) In the information processing method as described in Technique 2, the information related to the corrected individual risk includes control information for controlling the first mobile body and at least one of notification information for notifying the remote monitor, and outputs the generated control information and at least one of the notification information.

[0250] According to this, since the notification is made to the first moving object or the remote monitoring person, necessary and sufficient control can be performed on the first moving object.

[0251] (Technique 5) In the information processing method according to any one of Techniques 1 to 4, the sensing information is video information including one or more image information, and the image information is image information obtained by photographing the surroundings of the first moving object.

[0252] Thus, by using the image information, it is possible to easily detect two or more second moving objects and determine the positions and attributes of each of the two or more second moving objects. Furthermore, by using the image information, it is also possible to easily determine whether the two or more second moving objects are in a state with a predetermined correlation.

[0253] (Technique 6) In the information processing method according to Technique 3, the state information includes a moving speed, a moving direction, and a moving state of the first moving object.

[0254] According to this, since the moving speed, moving direction, and moving state of the first moving object included in the state information can be used, the individual risk can be calculated more accurately.

[0255] (Technique 7) As the information processing method described in Technique 3, when correcting the individual risk, if the individual risk of each of the two or more second moving bodies is different, correction is performed in a manner that reduces the individual risk of the second moving body with a high individual risk based on at least one of the types and attributes of each of the two or more second moving bodies.

[0256] According to this, since correction is performed so as to reduce the individual risk according to at least one of the type and the attribute of each second moving object, the individual risk can be calculated more accurately.

[0257] (Technique 8) In the information processing method as described in any one of Techniques 2 to 4, Technique 6, and Technique 7, when correcting the individual risk, a table including the type, attribute, and correction value of the second mobile body is used to correct the individual risk calculated for each of more than two second mobile bodies.

[0258] According to this, since the individual risk can be calculated using the table, the individual risk can be calculated more easily and with less calculation load.

[0259] (Technique 9) As the information processing method described in Technique 8, when correcting the individual risk, in a case where the detected two or more second moving bodies are two second moving bodies consisting of a person over a specified age and a child younger than the specified age, correction is performed according to the table in a manner that reduces the individual risk calculated for the second moving body of the child among the two second moving bodies.

[0260] In this way, the individual risk calculated for the two second moving bodies consisting of a person over the specified age and a child under the specified age can be corrected by taking into account the fact that there is a specified correlation. Accordingly, a more accurate risk assessment can be obtained for the two second moving bodies consisting of a person over the specified age and a child under the specified age.

[0261] (Technique 10) In the information processing method described in Technique 8, when correcting the individual risk, if each of the two or more detected second moving bodies is a child younger than a specified age, correction is performed according to the table in a manner that increases the individual risk calculated for each of the two or more second moving bodies.

[0262] In this way, the individual risks calculated for the two second moving objects consisting of children younger than the specified age can be corrected by taking the specified correlation into consideration. Thus, a more accurate risk assessment can be obtained for the two second moving objects consisting of children younger than the specified age.

[0263] (Technique 11) As the information processing method described in Technique 8, when correcting the individual risk, when the detected two or more second moving bodies are composed of an adult and two or more children younger than a specified age, correction is performed according to the table in a manner that reduces the individual risk calculated for each of the two or more second moving bodies of the two or more children among the two or more second moving bodies.

[0264] In this way, the individual risk calculated for two or more second moving bodies consisting of one adult and two or more children younger than the specified age can be corrected by taking into account the fact that there is a specified correlation. Accordingly, a more accurate risk assessment can be obtained for the two or more second moving bodies consisting of one adult and two or more children younger than the specified age.

[0265] (Technique 12) In the information processing method as described in any one of Techniques 1 to 11, when judging whether two or more second mobile bodies are in a state having the specified association, it is judged whether the two or more second mobile bodies are in a state having the specified association based on group judgment information of at least one of the distance, line of sight, physical connection, possession, movement, age, and sound between the two or more second mobile bodies.

[0266] In this way, by using at least one of the distance, line of sight, physical connection, possession, movement, age, and sound between two or more second mobile bodies as group judgment information, it is possible to easily determine whether two or more second mobile bodies have a specified correlation.

[0267] (Technique 13) In the information processing method as described in Technique 4, when generating at least one of the control information and the notification information, information for causing the first moving body to perform at least one of the following controls is generated as the control information according to the corrected value of the individual risk, and these controls refer to temporary stop control, avoidance action control, attention-drawing control through voice, and lighting control.

[0268] In this way, by generating at least one of control of temporarily stopping, control of performing avoidance action, control of using voice to attract attention, and control of lighting as control information of the first mobile body, the control information of the first mobile body can be generated easily and without load according to the value of the corrected individual risk.

[0269] (Technique 14) In the information processing method as described in Technique 4, when generating at least one of the control information and the notification information, an alarm information is generated as the notification information according to the corrected value of the individual risk, and the alarm information is information indicating an alarm prompting the remote monitor to watch the first moving body or operate the first moving body.

[0270] In this way, by generating alarm information indicating an alarm urging the remote monitor to watch or operate the first moving object as notification information, control information of the first moving object can be easily generated without load according to the corrected individual risk value.

[0271] (Technique 15) In the information processing method as described in Technique 14, the notification information includes information showing at least one of the size of an alarm display indicating the alarm, the color of the alarm display, whether the alarm display flashes, the volume of a notification sound for notifying the alarm, and the pitch of the notification sound.

[0272] Based on this, various notifications can be made as notification information.

[0273] (Other Embodiments, etc.)

[0274] Although the information processing method and the like involved in one or more methods are described above based on the implementation methods, the present disclosure is not limited to these implementation methods and variations. Without departing from the scope of the present disclosure, various modifications that can be thought of by those skilled in the art are performed on the implementation methods and variations, which are obtained by all means, which are included in the scope of the present disclosure.

[0275] In addition, in the above-mentioned embodiments, each component can be formed by dedicated hardware, or can be implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0276] In addition, the following cases are also included in the present disclosure.

[0277] (1) Specifically, the at least one device is a computer system composed of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor operates according to the computer program, so that the at least one device achieves its function. Here, the computer program is a combination of multiple command codes that issue instructions to the computer in order to achieve a specified function.

[0278] (2) A part or all of the components constituting at least one of the above-mentioned devices may be constituted by a system LSI (Large Scale Integration). A system LSI is a multifunctional LSI in which multiple components are integrated and manufactured on a single chip, and specifically is a computer system constituted by a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The microprocessor operates according to the computer program, so that the system LSI realizes its functions.

[0279] (3) A part or all of the components constituting at least one of the above-mentioned devices may be constituted by an IC card or a single module that can be installed and removed from the device. The IC card or single module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or single module may include the above-mentioned super-multifunctional LSI. The microprocessor operates according to the computer program, so that the IC card or single module realizes its respective functions.

[0280] (4) The present disclosure may be the methods shown above, a computer program for realizing these methods by a computer, or a digital signal composed of the computer program.

[0281] Furthermore, the present disclosure may also be a computer-readable recording medium recording a computer program or a digital signal, such as a floppy disk, a hard disk, a CD (Compact Disc: Optical Disk)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Furthermore, it may also be a digital signal recorded on these recording media.

[0282] Furthermore, the present disclosure may transmit computer programs or digital signals via electronic communication lines, wireless or wired communication lines, networks represented by the Internet, data broadcasting, and the like.

[0283] Furthermore, the program or digital signal may be recorded on a recording medium and then transferred, or the program or digital signal may be transferred via a network or the like, thereby being executed by another independent computer system.

[0284] Industrial Applicability

[0285] The present disclosure can be utilized in a method of outputting control information for a remote monitor to monitor or remotely operate a first moving object that is an operation target, and the like.

[0286] Explanation of symbols

[0287] 10 Information processing device

[0288] 11 Ministry of Communications

[0289] 12 Storage

[0290] 20 Processing Department

[0291] 21. Second moving object detection unit

[0292] 22. Second moving object position determining unit

[0293] 23 Individual Risk Calculation Department

[0294] 24 Group Relationship Judgment Department

[0295] 25 Risk Correction Department (Execution Department)

[0296] 26 Information Generation Department

[0297] 27 Output

[0298] A1, A2, C1, C2, C3, C4, C5 Second moving body

[0299] D1 Terminal

[0300] D11, M11 communication device

[0301] D12 Input Device

[0302] D13 Output Device

[0303] M1 First moving body

[0304] M12 Sensing Device

[0305] M13 Status information acquisition device

[0306] M14 Controls

[0307] M15 drive

[0308] N Network

[0309] R A 1. R C 1a, R C 1b, R C 2a, R C 2b, R C 3a, R C 3b, R C 4a, R C 4b, R C 5a, R C Area 5b

[0310] T1, T2, T3, T4, T5, T6 tables

Claims

1. An information processing method, which is an information processing method in an information processing device, In the information processing method, acquiring sensing information, wherein the sensing information is information obtained by sensing the surroundings of a first moving body that is autonomously moving, detecting a second moving object existing around the first moving object based on the sensing information, When two or more second moving bodies are detected to be present around the first moving body, determining whether the two or more second moving bodies are in a state having a predetermined correlation based on the sensing information, When two or more of the second moving bodies are in a state having the prescribed correlation, a prescribed operation is performed to prevent the two or more of the second moving bodies from contacting the first moving body.

2. The information processing method according to claim 1, In the information processing method, calculating individual risks based on a first position of the first moving body and second positions of each of the two or more second moving bodies, the individual risks indicating a risk possibility that each of the two or more second moving bodies will come into contact with the first moving body, When two or more of the second moving bodies are in a state having the prescribed association, the following operation is performed as the prescribed operation: correcting the individual risk calculated for each of the two or more second moving objects, Information on the corrected individual risk is notified to at least one of the first moving object and a terminal of a remote monitor, the remote monitor being a monitor who remotely monitors or operates the first moving object.

3. The information processing method according to claim 2, In the information processing method, obtaining state information related to the moving state of the first moving body, When two or more of the second moving bodies are in a state having the predetermined correlation, determining the second position and attribute of each of the two or more detected second moving objects, calculating the individual risk of each of the two or more second moving bodies based on the state information and the determined second positions of each of the two or more second moving bodies, Whether the two or more second moving objects are in a state having the prescribed correlation is determined based on the determined second positions and the attributes of the two or more second moving objects.

4. The information processing method according to claim 2, The information related to the corrected individual risk includes at least one of control information for controlling the first moving object and notification information for notifying the remote monitor, At least one of the generated control information and the generated notification information is output.

5. The information processing method according to any one of claims 1 to 4, The sensing information is video information including one or more image information, and the image information is image information obtained by photographing the surroundings of the first moving object.

6. The information processing method according to claim 3, The state information includes the moving speed, moving direction and moving state of the first moving object.

7. The information processing method according to claim 3, When correcting for the individual risks described, When the individual risk of each of the two or more second moving objects is different, correction is performed based on at least one of the type and attribute of each of the two or more second moving objects so as to reduce the individual risk of the second moving object having a high individual risk.

8. The information processing method according to any one of claims 2 to 4, When correcting for the individual risks described, The individual risk calculated for each of the two or more second moving objects is corrected using a table including the type, attribute, and correction value of the second moving object.

9. The information processing method according to claim 8, When correcting for the individual risks described, In a case where the two or more detected second moving bodies are two second moving bodies consisting of a person over a specified age and a child younger than the specified age, correction is performed according to the table in such a manner as to reduce the individual risk calculated for the child second moving body of the two second moving bodies.

10. The information processing method according to claim 8, When correcting for the individual risks described, When each of the two or more detected second moving objects is a child younger than a predetermined age, correction is performed based on the table so as to increase the individual risk calculated for each of the two or more second moving objects.

11. The information processing method according to claim 8, When correcting for the individual risks described, In a case where the detected two or more second moving bodies consist of an adult and two or more children younger than a specified age, correction is performed according to the table in such a manner that the individual risk calculated for each of the two or more second moving bodies of the two or more children among the two or more second moving bodies is reduced.

12. The information processing method according to any one of claims 1 to 4, When determining whether two or more of the second moving bodies are in a state having the prescribed correlation, Based on group judgment information of at least one of the distance, line of sight, physical connection, possession, action, age, and sound between two or more second moving bodies, it is determined whether two or more second moving bodies are in a state having the specified correlation.

13. The information processing method according to claim 4, When generating at least one of the control information and the notification information, According to the corrected value of the individual risk, information for causing the first moving body to perform at least one of the following controls is generated as the control information, and these controls are control of temporary stop, control of avoidance action, control through voice for attracting attention, and control of lighting.

14. The information processing method according to claim 4, When generating at least one of the control information and the notification information, According to the corrected value of the individual risk, warning information is generated as the notification information, the warning information being information indicating a warning for prompting the remote monitor to watch the first moving object or operate the first moving object.

15. The information processing method according to claim 14, The notification information includes information indicating at least one of a size of an alarm display indicating the alarm, a color of the alarm display, whether the alarm display flashes, a volume of a notification sound for notifying the alarm, and a pitch of the notification sound.

16. An information processing device comprising: an acquisition unit that acquires sensing information, wherein the sensing information is information obtained by sensing the surroundings of a first moving body that is autonomously moving; a detection unit that detects a second moving object existing around the first moving object based on the sensing information; a group relationship determination unit that, when detecting that there are two or more second moving bodies around the first moving body, determines whether the two or more second moving bodies are in a state of having a predetermined relationship based on the sensing information; as well as The execution unit executes a predetermined operation to prevent the two or more second moving bodies from contacting the first moving body when the two or more second moving bodies are in a situation having the predetermined correlation.

17. A program for causing a computer to execute the following processing: acquiring sensing information, wherein the sensing information is information obtained by sensing the surroundings of a first moving body that is autonomously moving, detecting a second moving object existing around the first moving object based on the sensing information, When two or more second moving bodies are detected to be present around the first moving body, determining whether the two or more second moving bodies are in a state having a predetermined correlation based on the sensing information, When two or more of the second moving bodies are in a state having the prescribed correlation, a prescribed operation is performed to prevent the two or more of the second moving bodies from contacting the first moving body.

Images