Information processing method, recording medium, and information processing device

Abstract

An information processing method includes the following processing: obtaining first information from multiple images reflecting a time series of an object, wherein the first information includes multiple information for at least one of the position and size of the object (S11), performing prediction processing based on the first information, in which second information including at least one of the position and size of the object at the next moment of the time series is predicted, recursively performing prediction processing based on the first information and the second information to predict the second information of the object at the next moment after the next moment (S12), performing recognition processing for recognizing the movement of the object based on the second information (S13), and determining the number of recursive prediction processing times based on the result of the recognition processing (S14).

Description

Technical Field

[0001] The present disclosure relates to an information processing method, a recording medium, and an information processing device. Background Art

[0002] In the past, a prediction technique has been proposed that uses past movement information of the vehicle and position information of the object in the past few frames projected by the camera to predict the future position of the object (e.g., Non-Patent Document 1). In the technique disclosed in Non-Patent Document 1, the position of the object at a certain moment in the future is predicted.

[0003] The following technology is also disclosed, which predicts the position of an object at a certain moment in the future, uses the predicted position to predict the position of the object at the next moment in a time series, and predicts the future positions of the object one by one by recursively executing this prediction process (for example, patent document 1).

[0004] (Prior art literature)

[0005] (Patent Document)

[0006] Patent Document 1 International Publication No. 2017 / 029924

[0007] (Non-patent literature)

[0008] Non-patent literature 1Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty (CVPR2018)

[0009] However, in the technology disclosed in Patent Document 1, the prediction process is recursively repeated, which increases the amount of calculation. In addition, the motion of an object is recognized from the position (at least one of the position and size on the image) of a plurality of objects. Summary of the invention

[0010] Therefore, the present disclosure provides an information processing method and the like capable of performing a prediction process of a calculation amount suitable for recognizing the motion of an object.

[0011] The information processing method involved in the present disclosure obtains first information from multiple images of a time series in which an object is projected, wherein the first information includes multiple information for at least one of the position and size of the object, performs prediction processing based on the first information, predicts second information including at least one of the position and size of the object at the next moment in the time series, recursively performs the prediction processing based on the first information and the second information to predict the second information of the object at the next moment after the next moment, performs recognition processing for recognizing the movement of the object based on the second information, and determines the number of recursive times of the prediction processing based on the result of the recognition processing.

[0012] In addition, these general or specific forms may be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0013] According to the information processing method and the like according to one aspect of the present disclosure, it is possible to perform prediction processing of the amount of calculation suitable for recognizing the movement of an object. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram for explaining that it is difficult to predict the behavior of an object by simply predicting the position of the object at a certain point in the future.

[0015] Figure 2 It is a diagram for explaining the prediction process which is performed recursively.

[0016] Figure 3 This is a block diagram showing an example of the configuration of an information processing device according to an embodiment.

[0017] Figure 4 This is a flowchart showing an example of an information processing method according to an embodiment.

[0018] Figure 5 This is a flowchart showing a specific example of the prediction process according to the embodiment. DETAILED DESCRIPTION

[0019] (The process of obtaining one form of the present disclosure)

[0020] First, use Figure 1 as well as Figure 2 The process of arriving at one embodiment of the present disclosure will be described.

[0021] Figure 1This diagram is used to explain that it is difficult to predict the movement of an object by simply predicting the position of the object at a certain point in the future.

[0022] Figure 2 It is a diagram for explaining the prediction process which is performed recursively.

[0023] As disclosed in non-patent document 1, the position of an object at a future time M can be predicted by using the past movement information of the own vehicle calculated by an odometer or the like and the position information of the object in the past few frames projected by the camera. Figure 1 As shown, the movement of an object after time M cannot be known based only on the predicted position of the object at time M, and it is difficult to predict the behavior or intention of an object after time M. Figure 1 In the example shown, the person at the predicted position at time M may run in front of the vehicle, may move in a direction parallel to the direction of travel of the vehicle, or may move away from the vehicle. It is difficult to predict the person's movements after time M.

[0024] Therefore, as disclosed in Patent Document 1, by recursively executing the prediction process, the movement of the object after time M can be predicted. Specifically, Figure 2 As shown, the position information of the object up to time N extracted from multiple images showing the object up to time N is used to predict the position information of the object at a specific time M in the future after time N. Next, the predicted position information of the object at time M is added to the data set, and the position information of the object at the next time after time M (for example, the time corresponding to the next frame of time M) is predicted using the data set. Next, the predicted position information of the object at time M+1 is added to the data set, and the position information of the object at the next time after time M+1 (for example, time M+2) is further predicted using the data set. By repeatedly performing this operation until time M+t, the position sequence of the object from time M to time M+t is predicted, that is, the movement of the object is predicted.

[0025] However, Patent Document 1 does not disclose to what extent the above-mentioned t is set, in other words, does not disclose to what extent the number of times the recursion of the prediction process is executed (hereinafter referred to as the number of recursions) is set. Therefore, when t is too small, the information of the position of the object (at least one of the position on the image and the size) for identifying the movement of the object will be insufficient. In other words, if the number of recursions is small, although the position of the object can be predicted, the amount of information such as the position of the object is insufficient for identifying the movement of the object. In addition, when t is too large, the amount of calculation used to identify the movement of the object will increase. In other words, if the number of recursions is large, the number of executions of the prediction process will increase, and the amount of calculation will increase.

[0026] Therefore, an information processing method involved in one form of the present disclosure obtains first information from multiple images in a time series that project an object, wherein the first information includes multiple information for at least one of the position and size of the object, performs prediction processing based on the first information, predicts second information including at least one of the position and size of the object at the next moment in the time series, recursively performs the prediction processing based on the first information and the second information to predict the second information of the object at the next moment after the next moment, performs recognition processing for recognizing the movement of the object based on the second information, and determines the number of recursive times of the prediction processing based on the result of the recognition processing.

[0027] In this way, by determining the number of recursions of the prediction process according to the recognition result of the object's action, the number of recursions can be determined so that it will not be insufficient for the recognition of the object's action. For example, when the number of recursions is too small, the information used to recognize the object's action will be insufficient, so that the recognition of the action (or the processing of the recognized action) can be suppressed. In addition, for example, the increase in the amount of calculation caused by too many recursions can be suppressed, or the failure of the recognition of the action caused by predicting unnecessary positions can be suppressed. In this way, since the number of recursions can be determined according to the recognition result of the object's action, both the recognition performance and the amount of calculation of the object's action can be established. In other words, the prediction processing of the amount of calculation suitable for the recognition of the object's action can be performed. In addition, in the action of the object, there are both actions that hope to make the number of recursions of the prediction process small and actions that hope to make the number of recursions of the prediction process large. For this, for example, a table showing the correspondence between the action of the object and the number of recursions can be prepared in advance, or a function for calculating the number of recursions according to the action of the object can be prepared in advance. Therefore, by recognizing the movement of the object, the number of recursions can be determined according to the recognized movement.

[0028] Furthermore, the recognition process may be performed based on the second information and the first information.

[0029] According to this, not only the predicted second information but also, for example, the first information prepared in advance for predicting the second information can be used for the recognition process, so the accuracy of the recognition process can be improved.

[0030] It is also possible that the movement of the object may include a moving direction.

[0031] As for the movement of the object, there are moving directions in which it is expected that the number of recursions in the prediction process will be small, and there are moving directions in which it is expected that the number of recursions in the prediction process will be large.

[0032] Therefore, for example, a table showing the correspondence between the moving direction of an object and the number of recursions can be prepared in advance, or a function for calculating the number of recursions according to the moving direction of the object can be prepared in advance, and the number of recursions suitable for the identified moving direction can be determined by identifying the moving direction of the object.

[0033] It may also be that the motion of the object includes a changing pattern of moving direction.

[0034] As the motion of the object, among the change patterns of the moving direction, there are both change patterns of the moving direction that are expected to make the number of recursions of the prediction process small and change patterns of the moving direction that are expected to make the number of recursions of the prediction process large. Therefore, for example, a table showing the correspondence between the change patterns of the moving direction of the object and the number of recursions can be prepared in advance, or a function for calculating the number of recursions according to the change patterns of the moving direction of the object can be prepared in advance, and the number of recursions suitable for the change pattern of the moving direction of the object can be determined by identifying the change pattern of the moving direction of the object.

[0035] It may also be that the movement of the object includes the speed of movement.

[0036] As the motion of the object, there are both moving speeds for which the number of recursive times of the predicted processing is small and moving speeds for which the number of recursive times of the predicted processing is large. Therefore, for example, a table showing the correspondence between the moving speed of the object and the number of recursive times can be prepared in advance, or a function for calculating the number of recursive times according to the moving speed of the object can be prepared in advance, and the number of recursive times suitable for the identified moving speed can be determined by identifying the moving speed of the object.

[0037] Furthermore, when determining the number of recursive operations of the prediction process, the number of recursive operations of the prediction process may be determined according to the degree of movement of the object recognized as a result of the recognition process.

[0038] According to this, it is possible to determine whether to increase or decrease the number of recursions of the prediction process according to the degree of the movement of the object.

[0039] For example, when determining the number of recursive times of the prediction processing, the greater the degree of movement of the object, the smaller the number of recursive times of the prediction processing is determined to be, or the smaller the degree of movement of the object, the larger the number of recursive times of the prediction processing is determined to be.

[0040] For example, when the degree of the movement of the object is large, the object will have a large impact on the surroundings, or the object will be greatly affected by the surroundings. Compared with the case where the degree of the movement of the object is small, by making the number of recursive times of the prediction process a small value, the prediction process can be terminated as soon as possible, and control corresponding to the action or intention of the object can be immediately performed. In addition, when the degree of the movement of the object is large, if the change of the movement of the object in the image of each frame increases, the number of recursive times of the prediction process for such movement increases, and the possibility of prediction of the movement of the object increases, resulting in a decrease in prediction accuracy. Therefore, when the degree of the movement of the object is large, compared with the case where the degree of the movement of the object is small, by making the number of recursive times of the prediction process a small value, the prediction accuracy of the movement of the object is not easily reduced. In addition, when the degree of the movement of the object is small, the object has a small impact on the surroundings or is less affected by the surroundings. Compared with the case where the degree of the movement of the object is large, by making the number of recursive times of the prediction process a large value, the prediction process can be continued to prepare for the future action or intention of the object. Furthermore, when the degree of the movement of the object is small, the change in the movement of the object reflected in the image of each frame is small, and even if the number of recursive times of the prediction process is increased for such a movement, the prediction accuracy is unlikely to decrease.

[0041] Furthermore, when determining the number of recursive operations of the prediction process, the number of recursive operations of the prediction process may be determined according to the type of movement of the object recognized as a result of the recognition process.

[0042] According to this, the number of recursive times of the prediction process can be determined according to the type of the movement of the object. Therefore, the processing in the above-mentioned table becomes easy.

[0043] Furthermore, when determining the number of recursive operations of the prediction process, the number of recursive operations of the prediction process may be determined according to a recognition rate of a motion of the object recognized as a result of the recognition process.

[0044] According to this, the number of recursions of the prediction process can be determined according to the recognition rate of the object's action. Therefore, when the recognition rate is low, the possibility of recognition can be increased by increasing the number of recursions. In addition, by determining the number of recursions for actions with a recognition rate above a threshold, it is possible to suppress the number of recursions from being determined according to erroneous actions.

[0045] Furthermore, a recording medium according to one aspect of the present disclosure can be read by a computer, and a program for causing the computer to execute the above-mentioned information processing method is recorded in the recording medium.

[0046] According to this, it is possible to provide a recording medium capable of performing a prediction process of a calculation amount suitable for object motion recognition.

[0047] Furthermore, an information processing device involved in one form of the present disclosure comprises: an acquisition unit, which acquires first information from a plurality of images in a time series in which an object is projected, wherein the first information includes a plurality of information regarding at least one of a position and a size of the object; a prediction unit, which performs a prediction process based on the first information, in which a second information including at least one of a position and a size of the object at a next moment in the time series is predicted, and the prediction process is recursively performed based on the first information and the second information to predict the second information of the object at a moment after the next moment; a recognition unit, which performs a recognition process for recognizing a movement of the object based on the second information; and a determination unit, which determines the number of recursive times of the prediction process based on a result of the recognition process.

[0048] This makes it possible to provide an information processing device capable of performing prediction processing of a calculation amount suitable for object motion recognition.

[0049] In addition, these general or specific forms may be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, or recording media.

[0050] The embodiments will be described in detail below with reference to the drawings.

[0051] In addition, the embodiments to be described below are all general or specific examples. The values, shapes, materials, components, configuration positions of components, connection methods, steps, and order of steps shown in the following embodiments are examples, and their purpose is not to limit the present disclosure.

[0052] (Implementation Method)

[0053] Below, using Figures 3 to 5 An implementation method will be described.

[0054] Figure 3 This is a block diagram showing an example of the configuration of the information processing device 10 according to the embodiment.

[0055] The information processing device 10 is a device that predicts the future movement of an object using a plurality of images of a time series showing the object obtained by taking a picture with a camera or the like. For example, the information processing device 10 may be a device mounted on a vehicle, or a server device capable of wirelessly communicating with the vehicle. In this case, the information processing device 10 predicts the future movement of a person, an animal, or a vehicle, etc., around the vehicle, which is an object. Furthermore, for example, the information processing device 10 may be a device installed indoors, or a server device capable of communicating with a device used indoors. In this case, the information processing device 10 predicts the future movement of a person, an animal, etc., which is an object, indoors.

[0056] The information processing device 10 includes: an acquisition unit 11, a prediction unit 12, an identification unit 13, a determination unit 14, an output unit 15, a first storage unit 16, and a second storage unit 17. The information processing device 10 is a computer including a processor and a memory. The memory is a ROM (Read Only Memory) and a RAM (Random Access Memory), etc., which can store programs executed by the processor. In addition, the first storage unit 16 and the second storage unit 17 are examples of memories. The first storage unit 16 and the second storage unit 17 can be implemented by the same memory, or by different memories. The acquisition unit 11, the prediction unit 12, the identification unit 13, the determination unit 14, and the output unit 15 are implemented by the processor and the like executing the program stored in the memory. In addition, the components constituting the information processing device 10 can also be distributed to multiple computers (server devices, etc.).

[0057] The first storage unit 16 stores a plurality of images in a time series showing an object. For example, when the information processing device 10 is a device mounted on a vehicle, or is a server device capable of wirelessly communicating with a vehicle, the first storage unit 16 accumulates a plurality of images captured so far by a camera mounted on the vehicle. In this case, the first storage unit 16 can store information used for the odometer, such as information such as the rotation angle of the tires of the vehicle so far. Furthermore, for example, when the information processing device 10 is a device installed indoors, or is a server device capable of communicating with a device used indoors, the first storage unit 16 accumulates a plurality of images captured so far by a camera installed indoors.

[0058] The second storage unit 17 stores the second information predicted by the prediction unit 12. The second information will be described later.

[0059] The functions of the acquisition unit 11, the prediction unit 12, the identification unit 13, the determination unit 14, and the output unit 15 are implemented by a processor or the like that executes the program. Figure 4 Provide explanation.

[0060] Figure 4 is a flowchart showing an example of an information processing method according to an embodiment. For example, the information processing method according to the embodiment is a method executed by a processor included in the information processing device 10. That is, in this case, Figure 4 This is also a flowchart showing the operations of the acquisition unit 11 , the prediction unit 12 , the identification unit 13 , the determination unit 14 , and the output unit 15 .

[0061] The acquisition unit 11 acquires the first information from a plurality of images in a time series in which the object is projected, and in the first information, at least one of the position and size of the object is included in a plurality (step S11). For example, the first storage unit 16 stores a plurality of images captured by a camera or the like so far, and the acquisition unit 11 acquires the first information from the plurality of images stored in the first storage unit 16. For example, the acquisition unit 11 extracts at least one of the position and size of the object from each of the plurality of images, and acquires the first information accordingly. The first information may include a plurality of only the position of the object, or may include a plurality of only the size of the object, or may include a plurality of each of the position and size of the object. The position of the object projected in each of the plurality of images can identify the movement of the object in two dimensions, and the size of the object projected in each of the plurality of images can identify the movement of the object in the depth direction. In addition, the position is not limited to the two-dimensional position on the image, and may also be a three-dimensional position. The three-dimensional position includes depth information that can identify the movement of the object in the depth direction, and in this case, the size may not be included in the first information.

[0062] In addition, when the information processing device 10 is a device mounted on a vehicle, or a server device capable of wirelessly communicating with a vehicle, the position or size of an object reflected in a camera mounted on the vehicle changes not only with the movement of the object itself, but also with the movement of the vehicle. Therefore, the acquisition unit 11 can also obtain information for the odometer. According to this, when the position or size of an object reflected in a camera mounted on the vehicle is affected by the movement of the vehicle, the change in the position of the vehicle can be estimated by the odometer, and the part affected by the change in the position or size of the object reflected in the camera mounted on the vehicle can be subtracted from the change in the position or size of the object reflected in the camera mounted on the vehicle. That is, the change in the position or size of the object included in the first information in each of the multiple images can be regarded as the change caused by the movement of the object itself. In addition, even if the information for the odometer is not used, the change in the position of the vehicle can be estimated from the change in the position or size of fixed objects such as buildings, signs or traffic lights reflected in the camera, so the acquisition unit 11 does not need to obtain information for the odometer.

[0063] The prediction unit 12 performs a prediction process for predicting the second information based on the first information, the second information including at least one of the position and size of the object at the next moment in the time series, and recursively performs the prediction process based on the first information and the second information to predict the second information of the object at the next moment after the next moment (step S12). Figure 2 The operation of the prediction unit 12 will be described.

[0064] First, the prediction unit 12 performs prediction processing for predicting the second information based on the first information, in which the first information is a plurality of images (for example, in the time series in which the object is projected) in which the object is projected. Figure 2 At least one of the position and size of the object until time N extracted from the first information (in the image from time 0 to time N) includes a plurality of information, and at least one of the position and size of the object at time M is included in the second information, and time M is the next time in the time series. Next, the prediction unit 12 recursively performs prediction processing based on the first information and the predicted second information of the object at time M, and predicts the second information of the object at the next time after time M (for example, time M+1). Recursively performing prediction processing means performing prediction processing, using the predicted second information of the object, performing prediction processing again, predicting the second information of the object at the next time, using the predicted second information to perform prediction processing again, predicting the second information at the next time, and repeating this process to predict the second information of future objects one by one.

[0065] For example, the second information predicted by the prediction unit 12 is stored in the second storage unit 17 and is added each time a recursive prediction process is performed. The acquisition unit 11 obtains the first information from the first storage unit 16 and obtains the second information from the second storage unit 17. The prediction unit 12 recursively performs prediction processing based on the first information and the second information obtained by the acquisition unit 11.

[0066] The recognition unit 13 performs recognition processing for recognizing the movement of the object based on the second information (step S13). For example, when performing the recognition processing, the recognition unit 13 uses the second information of at least two moments predicted by recursively performing the prediction processing. In addition, the recognition processing can be performed based on the second information and the first information. That is, although the recognition unit 13 recognizes the movement of the object based on at least the second information, it can also perform the recognition processing based on the first information in addition to the second information. In this case, the recognition unit 13 can use multiple positions, multiple sizes, or multiple positions and sizes for all images of the multiple images included in the first information, and can also use the position, size, or position and size of at least one image of the multiple images. For example, when performing the recognition processing, the recognition unit 13 uses the second information of at least one moment predicted by recursively performing the prediction processing, and the position, size, or position and size of at least one image of the multiple images included in the first information.

[0067] The movement of the object includes, for example, the moving direction. That is, the recognition unit 13 recognizes the direction in which the object moves. For example, the moving direction of the object can be obtained from two positions, two sizes, or both of the two positions and two sizes of the object.

[0068] Furthermore, the movement of the object includes, for example, a change pattern of the moving direction. That is, the recognition unit 13 recognizes how the moving direction of the object changes. For example, the recognition unit 13 recognizes whether the object is staggering or moving straight. For example, the change pattern of the moving direction of the object can be obtained from three or more positions, three or more sizes, or three or more positions and three or more sizes of the object.

[0069] Furthermore, the movement of the object includes, for example, the speed of movement. That is, the recognition unit 13 recognizes the speed of movement of the object. For example, the recognition unit 13 recognizes whether the object (for example, a person) is walking, running, or stopped. For example, the speed of movement of the object can be obtained from the time interval between two images corresponding to two positions, two sizes, or both of the two positions and two sizes of the object and the two positions or two sizes.

[0070] The decision unit 14 determines the number of recursive times of the prediction process based on the result of the recognition process (step S14). For example, the decision unit 14 determines the number of recursive times of the prediction process according to the degree of movement of the recognized object as a result of the recognition process. Specifically, the decision unit 14 determines the number of recursive times of the prediction process to be a smaller value when the degree of movement of the object is greater, or determines the number of recursive times of the prediction process to be a larger value when the degree of movement of the object is smaller. A large degree of movement of the object refers to, for example, a situation where the object is shaking, staggering, or running. A small degree of movement of the object refers to, for example, a situation where the object is walking straight, walking, or lying down.

[0071] When the degree of the object's movement is large, the object has a great impact on the surroundings or the object is greatly affected by the surroundings. Compared with the case where the degree of the object's movement is small, the number of recursive times of the prediction process is set to a small value, thereby being able to end the prediction process as early as possible and immediately perform control corresponding to the object's movement or intention. For example, when the object is shaking or staggering, the decision unit 14 determines the number of recursive times of the prediction process to be a small value. Accordingly, for example, a vehicle can immediately perform an action corresponding to the movement of the shaking or staggering object (such as slowing down or performing control to move away from the object). And, for example, when the object is running indoors, the decision unit 14 determines the number of recursive times of the prediction process to be a small value. Accordingly, for example, an indoor device can immediately perform an action corresponding to the intention of the running object (such as controlling to assist the rapid movement of the object).

[0072] Furthermore, when the degree of the object's motion is large, the change in the object's motion reflected in each frame of the image is large. If the number of recursive times of the prediction process for such motion is set to a large value, the possibility of the object's motion prediction increases, and the prediction accuracy decreases. Therefore, when the degree of the object's motion is large, the prediction accuracy of the object's motion is not easily reduced by setting the number of recursive times of the prediction process to a small value.

[0073] Furthermore, when the degree of the object's movement is small, the object has little impact on the surroundings or is less affected by the surroundings. Compared with the case where the degree of the object's movement is large, by setting the number of recursive times of the prediction process to a large value, the prediction process can be continuously performed to prepare for the future actions or intentions of the object. For example, when the object is moving straight, the decision unit 14 determines the number of recursive times of the prediction process to a large value. Accordingly, for example, a vehicle can cope with the future actions of the object moving straight. Furthermore, when the object is walking indoors, for example, the decision unit 14 determines the number of recursive times of the prediction process to a small value. Accordingly, for example, an indoor device can cope with the future intentions of the object that is walking.

[0074] Furthermore, when the degree of the movement of the object is small, the change in the movement of the object reflected in the image of each frame is small, and even if a large number of recursive times of the prediction process is set for such movement, the prediction accuracy is unlikely to decrease.

[0075] In addition, the decision unit 14 may determine the number of recursive times of the prediction process according to the moving direction of the object. For example, when the moving direction of the object is toward the front of the vehicle, the decision unit 14 determines the number of recursive times of the prediction process to be a small value. Accordingly, for example, the vehicle can immediately perform an action corresponding to the action of an object that may suddenly appear in front of the vehicle (for example, performing avoidance control, etc.). And, for example, when the moving direction of the object is toward a direction away from the vehicle, the decision unit 14 determines the number of recursive times of the prediction process to be a small value. Accordingly, for example, the vehicle can respond to the future action of an object that is about to move away from the vehicle.

[0076] The output unit 15 outputs the prediction result, which is the result of predicting the behavior of the object by executing the prediction process for the determined number of recursions (step S15). The output unit 15 outputs the prediction result, for example, to a vehicle or indoor device.

[0077] Next, use Figure 5 A specific example in which the prediction process is recursively executed will be described. Figure 5 This is a flowchart showing a specific example of the prediction process according to the embodiment.

[0078] First, the prediction unit 12 initializes the estimated time t and initializes x to 1 (step S21). The estimated time t is the time for the recursively executed prediction process, which corresponds to the above-mentioned number of recursions. For example, when the estimated time t is 1 minute, when the prediction process is performed once every 1 second, the estimated time of 1 minute corresponds to 60 recursions. The estimated time t is, for example, a variable, into which a value corresponding to the recognition result of the recognition unit 13 is substituted. The initial value of the estimated time t is appropriately determined according to the purpose of the information processing device 10. x is used to determine whether to terminate the recursively executed prediction process, for example, it is a variable that is incremented each time the prediction process is executed. That is, x represents the time or number of prediction processes performed so far.

[0079] Next, the prediction unit 12 predicts the second information of the object at time N+x (step S22). When the prediction unit 12 predicts the first time after x is initialized, that is, when x is 1, the first information up to time N is used to predict the second information. When the processing of step S22 is performed next and thereafter, the prediction processing is recursively performed using the second information predicted so far, and the second information of the object at the next time after the last prediction time is predicted one by one.

[0080] Next, the recognition unit 13 recognizes the movement of the object based on the prediction result of the prediction unit 12 (step S23). At this time, the recognition unit 13 may also use the first information up to time N to recognize the movement of the object. When recognizing the movement of the object, not only the information of the predicted future position or size of the object is used, but also the information of the position or size of the object in the past is used, so that the accuracy of the recognition process can be improved. In addition, the recognition unit 13 may also use the second information predicted recently to recognize the latest movement of the object.

[0081] Next, the decision unit 14 updates the estimated time t according to the recognition result of the recognition unit 13 (step S24). For example, in the information processing device 10, a table corresponding to the estimated time t is stored according to each recognition result (for example, the type of the object's movement), and the decision unit 14 updates the estimated time t so far to the estimated time t corresponding to the recognition result by comparing the recognition result with the table. For example, when the recognition result is the same as the recognition result in the last prediction processing, the estimated time t is maintained, in other words, it is updated to the same value. For example, a recognition result with a recognition rate above a threshold is used in the update of the estimated time t. In addition, the recognition result and the estimated time t may not be a one-to-one relationship, and the estimated time t can be calculated from a function corresponding to the recognition rate of the recognition result (specifically, the recognition rate of the object's movement). For example, when it is recognized that the object is shaking, the estimated time t can be calculated using a function in which the higher the recognition rate of the shaking, the smaller the estimated time t.

[0082] In this way, the decision unit 14 can determine the number of recursive times of the prediction process according to the type of movement of the object to be identified as a result of the recognition process. Accordingly, the number of recursive times of the prediction process can be determined according to the type of movement of the object. Therefore, the processing in the above-mentioned table becomes easy. In addition, the decision unit 14 can also determine the number of recursive times of the prediction process according to the recognition rate of the movement of the object to be identified as a result of the recognition process. Accordingly, the number of recursive times of the prediction process can be determined according to the recognition rate of the movement of the object. Therefore, in the case of a low recognition rate, by increasing the number of recursive times, the possibility of recognition can be increased. In addition, by determining the number of recursive times for an action whose recognition rate becomes above a threshold, it is possible to suppress the determination of the number of recursive times according to an erroneous action.

[0083] Next, the prediction unit 12 determines whether x is smaller than the estimated time t (step S25). That is, the prediction unit 12 determines whether the time of the prediction process performed so far has reached the estimated time t.

[0084] When the prediction unit 12 determines that x is less than the estimated time t ("Yes" in step S25), x is incremented (step S26) to continue the recursive prediction process. That is, the process after step S22 is executed again. In addition, when the prediction unit 12 determines that x is greater than the estimated time t ("No" in step S25), the recursive prediction process is terminated, and the output unit 15 outputs the prediction result of the object's movement obtained by the recursive prediction process (step S27). Based on this, a vehicle or indoor device, etc. that has obtained the prediction result of the object's movement can perform control corresponding to the object's action or intention according to the object's future movement.

[0085] In addition, the estimated time t is not particularly limited, as long as it can determine the timing of the prediction process to end the recursive execution, that is, as long as it corresponds to the number of recursions mentioned above. For example, the estimated time or the number of frames can be used instead of the estimated time t. In this case, the value corresponding to the estimated time or the number of frames is substituted into x.

[0086] As described above, by determining the number of recursions of the prediction process based on the recognition result of the object's motion, the number of recursions can be determined to be a number that is not insufficient for the recognition of the object's motion. For example, if the number of recursions is too small, the information for recognizing the object's motion will be insufficient, and the recognition of the motion (or the processing of the recognized motion) will fail. This situation can be suppressed. In addition, for example, the increase in the amount of calculation caused by too many recursions can be suppressed, or the failure of motion recognition caused by predicting unnecessary positions can be suppressed. In this way, since the number of recursions can be determined according to the recognition result of the object's motion, the recognition performance and the amount of calculation of the object's motion can be established. In other words, a prediction process with an amount of calculation suitable for the recognition of the object's motion can be performed.

[0087] (Other Embodiments)

[0088] The above is an explanation of the information processing method and information processing device 10 involved in one or more forms of the present disclosure based on the embodiments, and the present disclosure is not limited to these embodiments. Forms obtained by performing various modifications that can be thought of by those skilled in the art on the embodiments within the scope of the present disclosure, and forms formed by combining constituent elements in different embodiments are all included in the scope of one or more modes of the present disclosure.

[0089] The present disclosure can be implemented as a program that causes a processor to execute the steps included in the information processing method. Furthermore, the present disclosure can be implemented as a non-transitory computer-readable recording medium such as a CD-ROM on which these programs are recorded.

[0090] For example, when the present disclosure is implemented by a program (software), each step is executed by executing the program using hardware resources such as a computer's CPU, memory, and input / output circuits. That is, the CPU obtains data from the memory or the input / output circuit and performs calculations, and each step is executed by outputting the calculation results to the memory or the input / output circuit.

[0091] In addition, in the above-mentioned embodiment, each component included in the information processing device 10 may be constituted by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0092] Part or all of the functions of the information processing device 10 involved in the above-mentioned embodiment can be implemented as an LSI of an integrated circuit. These can be made into a single chip separately, or part or all of them can be made into a single chip. In addition, integrated circuitization is not limited to LSI, and can also be implemented by a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connection or setting of the circuit unit inside the LSI.

[0093] Furthermore, various modifications obtained by performing changes in the scope that can be conceived by a person skilled in the art on the embodiments of the present disclosure are all included in the present disclosure without departing from the gist of the present disclosure.

[0094] The information processing method and the like of the present disclosure can be applied to a device mounted on a vehicle or the like that performs control corresponding to surrounding objects, for example.

[0095] Explanation of symbols

[0096] 10 Information processing device

[0097] 11. Acquisition Department

[0098] 12 Forecasting Department

[0099] 13 Identification

[0100] 14 Decision-making Department

[0101] 15 Output

[0102] 16. Storage Unit 1

[0103] 17 Second storage unit

Claims

1. An information processing method, From a plurality of time-series images showing an object, first information including at least one of a position and a size of the object is obtained for each of the plurality of images; Based on the first information, a prediction process is performed in which second information including at least one of the position and the size of the object at the next moment in the time series is predicted, recursively performing the prediction process based on the first information and the second information to predict the second information of the object at the next moment after the next moment, performing recognition processing for recognizing the movement of the object based on the second information at at least two moments, The number of recursive operations of the prediction process is determined based on the result of the recognition process.

2. The information processing method according to claim 1, The recognition process is also performed based on the first information.

3. The information processing method according to claim 1, The motion of the object includes a moving direction.

4. The information processing method according to claim 3, The motion of the object includes a changing pattern of movement direction.

5. The information processing method according to claim 1, The motion of the object includes the speed of movement.

6. The information processing method according to claim 1, When determining the number of recursive operations of the prediction process, the number of recursive operations of the prediction process is determined according to the degree of movement of the object recognized as a result of the recognition process.

7. The information processing method according to claim 6, When determining the number of recursions of the prediction process, The greater the degree of the movement of the object, the smaller the number of recursions of the prediction process is determined to be, or The smaller the degree of the movement of the object, the larger the number of recursions of the prediction process is determined to be.

8. The information processing method according to claim 1, When determining the number of recursive operations of the prediction process, the number of recursive operations of the prediction process is determined according to the type of movement of the object recognized as a result of the recognition process.

9. The information processing method according to any one of claims 1 to 8, When determining the number of recursive times of the prediction process, the number of recursive times of the prediction process is determined according to a recognition rate of the motion of the object recognized as a result of the recognition process. 10 . A recording medium capable of being read by a computer, wherein a program is stored in the recording medium, the program being for causing the computer to execute the information processing method according to claim 1 .

11. An information processing device comprising: an acquisition unit that acquires first information including at least one of a position and a size of the object for each of a plurality of images in a time series in which the object is projected; A prediction unit, based on the first information, performs a prediction process, in which second information including at least one of the position and the size of the object at the next moment of the time series is predicted, and based on the first information and the second information, the prediction process is recursively performed to predict the second information of the object at the next moment after the next moment; a recognition unit that performs recognition processing for recognizing the movement of the object based on the second information at at least two moments; as well as The determination unit determines the number of recursions of the prediction process based on a result of the recognition process.

Images