Information processing device and method, program, and storage medium

Abstract

To provide an information processing device that can control the relationship between the reliability of subject recognition and the processing load so as to match a user's intention.SOLUTION: An information processing device includes an acquisition unit that acquires a first video image, an extraction unit that extracts a portion of an image from the first video image, a recognition unit that performs recognition processing to recognize a subject included in the first video image, and a control unit that controls the recognition unit to perform recognition processing using some of the images if the processing load required for the recognition processing is to be suppressed.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for recognizing a subject from an image. [Background technology]

[0002] In recent years, AI technology centered on machine learning has been actively developed. For example, technology that automatically recognizes information about subjects (e.g., person names, object names, gender, age, etc.) contained in videos and still images taken with a camera is being developed. It is conceivable that such technology could be used to add tags to videos taken with a camera.

[0003] Patent Document 1 discloses a method for reducing the processing load required for object recognition by using an image obtained by reducing the amount of data from an image to be subjected to object recognition for the object recognition. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5989781 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the system described in Patent Document 1, images with reduced data volume are used for object recognition, so that highly reliable recognition results are not necessarily obtained. On the other hand, in order to obtain highly reliable recognition results, images with unreduced data volume must be used for object recognition, which increases the processing load, such as the time required for object recognition, battery consumption, and communication volume. In such a situation where there is a trade-off between the reliability of object recognition and the high processing load, it is considered that there are cases where the user wants to prioritize reducing the processing load and cases where the user wants to prioritize obtaining highly reliable recognition results.

[0006] For example, when a user is performing tag display or search operations on a camera, the user is likely to want to obtain the tag (subject recognition result) immediately. Also, when the camera's battery is low, the user is likely to want to conserve battery power as much as possible. Also, assuming that the user pays communication fees according to the amount of communication used by the camera, the user is likely to want to conserve communication power as much as possible when the accumulated amount of communication is large. In these cases, the user is likely to want to prioritize reducing the processing load.

[0007] On the other hand, when none of the above conditions are met, it is considered that the user would prefer to obtain a more reliable recognition result.

[0008] However, with conventional techniques, it has not been possible to control the relationship between the reliability of subject recognition and the processing load in a manner that meets the user's intentions.

[0009] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide an information processing device that can control the relationship between the reliability of subject recognition and the processing load to suit the user's intentions. [Means for solving the problem]

[0010] An information processing device according to the present invention comprises an acquisition means for acquiring a first moving image, an extraction means for extracting a portion of an image from the first moving image, a recognition means for performing a recognition process for recognizing a subject included in the first moving image, a determination means for determining whether or not a processing load required for the recognition process should be reduced, and a control means for controlling the recognition means to perform the recognition process using the portion of the image extracted from the first moving image when the determination means determines that the processing load required for the recognition process should be reduced, and for controlling the recognition means to perform the recognition process using the first moving image when the determination means determines that the processing load required for the recognition process should not be reduced, is before Accepts operations that utilize the results of the recognition process. In this case, the processing load required for the recognition process should be reduced. It is characterized by making a judgment. Effect of the Invention

[0011] According to the present invention, it is possible to control the relationship between the reliability of subject recognition and the processing load so as to meet the user's intention. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing the configuration of a system including an information processing device and a server device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a conceptual diagram showing an example of management information stored in an information processing device. [Diagram 3] FIG. 4 is a sequence diagram illustrating an overview of processing performed by an information processing device and a server device. [Figure 4] 4 is a flowchart showing the operation of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0014] <System configuration> FIG. 1 is a block diagram showing the configuration of a system including an information processing device and a server device according to an embodiment of the present invention.

[0015] In FIG. 1, an information processing device 100 is a device having an imaging function and a communication function, such as a digital camera, a smartphone terminal, a tablet terminal, or a game device.

[0016] The control unit 101 is a control unit that controls each unit of the information processing device 100 according to input signals and programs, and is implemented by, for example, a Central Processing Unit (CPU). Note that instead of the control unit 101 controlling the entire information processing device 100, the entire device may be controlled by multiple hardware devices sharing the processing.

[0017] The memory 102 is used as a buffer memory for temporarily storing various data, a working area for the control unit 101, etc. The non-volatile memory 103 is an electrically erasable and recordable non-volatile memory, and stores programs executed by the control unit 101, etc.

[0018] The operation unit 104 is used to receive instructions from a user for the information processing device 100. The operation unit 104 includes, for example, a power button for the user to instruct ON / OFF of the power supply of the information processing device 100 and an operation button for instructing ON / OFF of a communication function. In addition, the operation unit 104 also includes a touch panel formed on the display unit 105 described later.

[0019] The display unit 105 displays a GUI (Graphical User Interface) for interactive operations. The display unit 105 does not necessarily have to be built into the information processing device 100. The information processing device 100 only needs to have at least a display control function for controlling the display contents.

[0020] The storage medium 106 can store various data. The storage medium 106 may be configured to be detachable from the information processing device 100, or may be built into the information processing device 100. In other words, it is sufficient that the information processing device 100 has at least a means for accessing the storage medium 106.

[0021] The imaging unit 107 is configured to include, for example, an optical lens unit having an optical system and a drive control unit that drives and controls the aperture, zoom, focus, etc., and an imaging element for converting light (image) introduced through the optical lens unit into an electrical image signal. Under the control of the control unit 101, the imaging unit 107 converts the subject image formed by the optical lens unit included in the imaging unit 107 into an electrical signal using the imaging element, performs noise reduction processing, etc., and outputs digital image data.

[0022] The communication unit 110 is a communication unit for realizing communication with other devices. The communication unit 110 is composed of, for example, an antenna for wireless communication and a communication controller for processing wireless signals, and realizes public wireless communication according to standards such as W-CDMA (UMTS) and LTE (Long Term Evolution). The control unit 101 controls the communication unit 110 and communicates with the server device 120 via the public line 140.

[0023] The server device 120 is a device having an object recognition function and a communication function, such as a personal computer.

[0024] The control unit 121 is a control unit that controls each unit of the server device 120 according to an input signal or a program, and is implemented by, for example, a Central Processing Unit (CPU). Note that instead of the control unit 121 controlling the entire server device 120, the entire device may be controlled by multiple hardware devices sharing the processing. The control unit 121 has a recognition processing function that recognizes subject information (e.g., person's name, action, emotion, etc.) included in a video or still image received via the communication unit 130 described below.

[0025] The memory 122 is used as a buffer memory for temporarily storing various data, a working area for the control unit 121, etc. The non-volatile memory 123 is an electrically erasable and recordable non-volatile memory, and stores programs executed by the control unit 121, etc.

[0026] The operation unit 124 is used to receive instructions from a user for the server device 120. The operation unit 124 includes, for example, a power button that allows the user to instruct the server device 120 to power on / off. Note that the operation unit 124 does not necessarily have to be built into the server device 120. It is sufficient that the server device 120 has at least a control function for controlling the contents of the operation.

[0027] The display unit 125 displays a GUI (Graphical User Interface) for interactive operations. Note that the display unit 125 does not necessarily have to be built into the server device 120. The server device 120 only needs to have at least a display control function for controlling the display contents.

[0028] The storage medium 126 can store various data. The storage medium 126 may be configured to be detachable from the server device 120, or may be built into the server device 120. In other words, the server device 120 only needs to have at least a means for accessing the storage medium 126.

[0029] The communication unit 130 is a communication unit for realizing communication with other devices. The communication unit 130 is composed of, for example, a communication controller for processing communication signals, and realizes wired communication in accordance with the IEEE802.3 standard. The control unit 121 controls the communication unit 130 and communicates with the information processing device 100 via the public line 140.

[0030] <System operation overview> Next, an overview of the operation of the system in this embodiment will be described. Fig. 2 is a diagram showing an example of information that the information processing device 100 stores in the storage medium 106 in order to realize the system in this embodiment.

[0031] Management information 200 is information related to moving images (moving images) captured by information processing device 100, and is composed of a combination of at least a file name 201, a face detection time stamp 202, a motion detection time stamp 203, a tag 204, and a recognition method 205. Management information 200 is stored in association with the captured moving image.

[0032] File name 201 is the file name of the video captured by information processing device 100. Face detection timestamp 202 is a timestamp indicating the location where a face was detected in the video. Motion detection timestamp 203 is a timestamp indicating the location where motion was detected in the video. Tag 204 is information about a subject included in the video. Recognition method 205 is information indicating the subject recognition method used to obtain tag 204.

[0033] Fig. 3 is a sequence diagram showing a processing flow in the system of this embodiment. In Fig. 3, it is assumed that the information processing device 100 and the server device 120 are in a state in which a connection is established between them via the communication units 110 and 130. It is assumed that the communication processing shown in Fig. 3 is performed via the communication units 110 and 130 of each device. It is assumed that the processing shown in steps S309 to S312 in Fig. 3 requires a smaller load (time, battery consumption, communication volume) than the processing shown in steps S315 to S317.

[0034] When the control unit 101 of the information processing device 100 accepts an operation to start moving image shooting via the operation unit 104 in step S301, the control unit 101 stores image data acquired by the imaging unit 107 in the memory 102 in step S302.

[0035] Thereafter, if the control unit 101 of the information processing device 100 detects a face in the image data stored in step S302, then in step S303, it stores a face detection timestamp in the memory 102. Furthermore, if the control unit 101 of the information processing device 100 detects a movement in the image data stored in step S302, then in step S304, it stores a movement detection timestamp in the memory 102.

[0036] In step S305, the control unit 101 of the information processing device 100 repeats the processes of steps S302 to S304 until an operation to end moving image shooting is received via the operation unit 104.

[0037] When the control unit 101 of the information processing device 100 receives an operation to end video shooting via the operation unit 104 in step S305, in step S306, it converts the multiple image data stored in step S302 into a predetermined video format and saves it on the storage medium 106.

[0038] Next, the control unit 101 of the information processing device 100 stores the following information in the management information 200 in step S307. File name 201: The file name saved in step S306 Face detection time stamp 202: The face detection time stamp stored in step 303 Motion detection timestamp 203: the motion detection timestamp stored in step 304 Next, in step S308, the control unit 101 of the information processing device 100 accepts an operation via the operation unit 104 requesting to display tags for the video saved in step S306, and in step S309 extracts a partial video from the corresponding video. Here, the control unit 101 of the information processing device 100 refers to the management information 200, and generates a video by extracting several frames before and after the face detection timestamp 202 and the motion detection timestamp 203 from the corresponding video.

[0039] Control unit 101 of information processing device 100 repeats the processes shown in steps S310 to S312 the number of times corresponding to the number of moving images extracted in step S309. In step S310, control unit 101 of information processing device 100 transmits an object recognition request to server device 120. Control unit 101 of information processing device 100 includes the moving images extracted in step S309 in the object recognition request.

[0040] When the control unit 121 of the server device 120 receives the object recognition request in step S310, the control unit 121 recognizes object information (e.g., person name, action, emotion, etc.) included in the received video in step S311. Then, in step S312, the control unit 121 of the server device 120 transmits an object recognition response to the information processing device 100. The control unit 121 of the server device 120 includes the object recognition result recognized in step S311 in the object recognition response.

[0041] When the control unit 101 of the information processing device 100 receives the object recognition response S312, the control unit 101 additionally stores the following information in the management information 200 in step S313. Tag 204: All object recognition results received in step S312 ·Recognition method 205: “Part” Next, in step S314, the control unit 101 of the information processing device 100 displays on the display unit 105 the tag of the subject recognition result received in step S312.

[0042] In this way, the control unit 101 of the information processing device 100 determines that the processing load (time here) should be reduced by accepting the tag display operation in S308, and uses a video in which a predetermined portion is extracted from the video for object recognition. This makes it possible to shorten the time until the user obtains the object recognition result.

[0043] Next, in step S315, when the control unit 101 of the information processing device 100 accepts an operation to end the tag display of the video via the operation unit 104, in step S316, the control unit 101 of the information processing device 100 transmits an object recognition request to the server device 120. The control unit 101 of the information processing device 100 includes the video saved in step S306 in the object recognition request.

[0044] Upon receiving the object recognition request in step S316, control unit 121 of server device 120 recognizes object information (e.g., person name, action, emotion, etc.) included in the received video in step S317. Thereafter, control unit 121 of server device 120 transmits an object recognition response to information processing device 100 in step S318. Control unit 121 of server device 120 includes the object recognition result of step S317 in the object recognition response.

[0045] When the control unit 101 of the information processing device 100 receives the object recognition response in step S318, the control unit 101 additionally stores the following information in the management information 200 in step S319. Tag 204: Object recognition result received in step S318 ·Recognition method 205: “Whole” In this way, the control unit 101 of the information processing device 100 determines that it is not necessary to suppress the processing load (time here) by accepting the tag display end operation, and uses the entire video for object recognition. This allows the user to obtain a more reliable object recognition result.

[0046] <Operation of each device> Next, the detailed operation of the information processing device 100 for realizing the above operations will be described with reference to FIG.

[0047] 4 is a flowchart showing the operation of the information processing device 100 of this embodiment. The process shown in this flowchart is realized by the control unit 101 of the information processing device 100 controlling each unit of the information processing device 100 in accordance with an input signal or a program. Also, it is assumed that the information processing device 100 is in a state where a connection is established with the server device 120 via the communication unit 110.

[0048] In step S401, the control unit 101 determines whether or not an operation to end the operation of the information processing device 100 (for example, a power OFF operation) has been accepted via the operation unit 104. If the control unit 101 determines that the operation has been accepted, it ends the processing of this flowchart, and if not, it proceeds to step S402.

[0049] In step S402, the control unit 101 determines whether or not an operation to start video shooting has been accepted via the operation unit 104. If the control unit 101 determines that the operation has been accepted, the process proceeds to step S403, and if not, the process proceeds to step S411.

[0050] In step S403, the control unit 101 stores the image data acquired by the imaging unit 107 in the memory 102. This step corresponds to the process of step S302 in FIG.

[0051] In step S404, the control unit 101 determines whether or not a face is included in the image data stored in step S403. If the control unit 101 determines that a face is included, the process proceeds to step S405, and if not, the process proceeds to step S406.

[0052] In step S405, the control unit 101 stores the time stamp recorded in step S404 in the memory 102. This step corresponds to the process in step S303 in FIG.

[0053] In step S406, the control unit 101 determines whether or not the image data stored in step S403 includes motion. If the control unit 101 determines that motion is included, the process proceeds to step S407. If the control unit 101 determines that motion is not included, the process proceeds to step S408.

[0054] In step S407, the control unit 101 stores the time stamp recorded in step S406 in the memory 102. This step corresponds to the process in step S304 in FIG.

[0055] In step S408, the control unit 101 determines whether or not an operation to end video shooting has been accepted via the operation unit 104. If the control unit 101 determines that the operation has been accepted, the process proceeds to step S409, and if not, the process returns to step S403.

[0056] In step S409, the control unit 101 converts the multiple pieces of image data stored in step S403 into a predetermined video format and stores the converted data in the storage medium 106. This step corresponds to the process in step S306 in FIG.

[0057] In step S410, the control unit 101 stores the following information in the management information 200: File name 201: The file name saved in step S409 Face detection timestamp 202: The timestamp stored in step 405 Motion detection timestamp 203: The timestamp stored in step 407 This step corresponds to the process in step S307 in FIG.

[0058] In step S411, the control unit 101 determines whether or not the processing load should be reduced. The control unit 101 determines that the processing load should be reduced when any of the following conditions is met: When an operation to display or search for the tag 204 is received via the operation unit 104 - When the battery level is below a certain level When the cumulative amount of communication via the communication unit 110 is equal to or greater than a predetermined amount For example, when a user is performing tag display or search operations on the information processing device 100, it is considered that the user wants to obtain the tag (subject recognition result) immediately. Also, when the remaining battery charge of the information processing device 100 is low, it is considered that the user wants to conserve the remaining battery charge as much as possible. Also, assuming that the user pays communication charges according to the amount of communication of the information processing device 100, it is considered that the user wants to conserve the amount of communication as much as possible when the accumulated amount of communication is large. On the other hand, when none of the above conditions are met (other than the case of the above conditions), it is considered that the user wants to obtain a more reliable subject recognition result.

[0059] If the control unit 101 determines that the processing load should be reduced, it repeats the processes of steps S412 to S415 for videos that have not been tagged with tags 204. Otherwise, the control unit 101 repeats the processes of steps S416 to S418 for videos whose recognition method 205 is not "whole". Note that the former process requires a smaller processing load (time, battery consumption, communication volume) than the latter process.

[0060] In step S412, the control unit 101 extracts a partial moving image from the moving image. Here, the control unit 101 references the management information 200, and generates a moving image by extracting several frames before and after the face detection timestamp 202 and the motion detection timestamp 203 from the corresponding moving image. This step corresponds to the processing of step S309 in FIG. 3.

[0061] The control unit 101 repeats the processes of steps S413 to S414 for the moving image extracted in step S412.

[0062] In step S413, control unit 101 transmits an object recognition request to server device 120 via communication unit 110. Control unit 101 includes the video extracted in step S412 in the object recognition request. This step corresponds to the process of step S310 in FIG.

[0063] In step S414, the control unit 101 determines whether or not an object recognition response has been received from the server device 120 via the communication unit 110. If the control unit 101 determines that an object recognition response has been received, the control unit 101 proceeds to the next step (step S413 if there is an unrecognized video, and step S415 if there is not), and if it determines that there is not an unrecognized video, the control unit 101 repeats the process of step S414 again.

[0064] In step S415, the control unit 101 additionally stores the following information in the management information 200 for the moving image that is the subject recognition target. Tag 204: All object recognition results received in step S414 ·Recognition method 205: “Part” This step corresponds to the process in step S313 in FIG.

[0065] In this way, when the control unit 101 of the information processing device 100 determines that the processing load (time, battery consumption, communication volume, etc.) should be reduced, the control unit 101 uses a video obtained by extracting a predetermined portion from the video for object recognition. This reduces the time until the user obtains the object recognition result, and can reduce battery consumption and communication volume.

[0066] In step S416, control unit 101 transmits an object recognition request to server device 120 via communication unit 110. Control unit 101 includes a moving image to be recognized as an object in the object recognition request. This step corresponds to the process of step S316 in FIG. 3.

[0067] In step S417, the control unit 101 determines whether or not an object recognition response has been received from the server device 120 via the communication unit 110. If the control unit 101 determines that the object recognition response has been received, the process proceeds to step S418, and if the control unit 101 determines that the object recognition response has not been received, the process repeats the process of step S417 again.

[0068] In step S418, the control unit 101 additionally stores the following information in the management information 200 for the moving image that is the subject recognition target. Tag 204: Object recognition result received in step S417 ·Recognition method 205: “Whole” This step corresponds to the process in step S319 in FIG.

[0069] In this way, when the control unit 101 of the information processing device 100 determines that it is not necessary to suppress the processing load (time, battery consumption, communication amount, etc.), it uses the entire video for object recognition. This allows the user to obtain a more reliable object recognition result.

[0070] The operation of the server device 120 is simple: when it receives an object recognition request from the information processing device 100, it performs object recognition on the video included in the object recognition request and transmits an object recognition response to the information processing device 100. Therefore, illustration of the operation is omitted.

[0071] As described above, the information processing device of this embodiment determines whether the processing load required for object recognition should be reduced, and switches between using an image extracted from a part of a video or using the entire video for object recognition. This allows the device to prioritize reducing the processing load when the processing load required for object recognition should be reduced, and to prioritize obtaining a highly reliable recognition result when the processing load should not be reduced.

[0072] (Modification of the embodiment) In the above embodiment, the object recognition result is obtained from an external server device, but the information processing device may have an object recognition function inside. That is, when the processing load (time, battery consumption, etc.) needs to be reduced, object recognition may be performed using a partially extracted video, and otherwise object recognition may be performed using the entire video.

[0073] Furthermore, in the above embodiment, a configuration in which a partially extracted moving image is used to suppress the processing load has been exemplified, but a configuration in which a still image (still picture) is extracted may also be used.

[0074] In the above embodiment, the same server device performs object recognition for all the partially extracted videos, but the video where the face is detected and the video where the movement is detected may be performed by different server devices. For example, if some server devices are good at object recognition and others are not, a more reliable object recognition result can be obtained by selecting a server device that is good at it.

[0075] In addition, in the above embodiment, the algorithm for object recognition has been described without any particular limitation, but a function for recognizing an object using machine learning may be implemented. In this way, the object may be recognized more accurately. Specific algorithms for machine learning include nearest neighbor method, naive Bayes method, decision tree, support vector machine, etc. In addition, deep learning that uses a neural network to generate features for learning and connection weighting coefficients by itself may also be mentioned. Any of the above algorithms that can be used can be appropriately applied to this embodiment.

[0076] For example, when implementing a function to recognize a subject using a learning model such as deep learning, an image of a person is used as input data, and a learned model is created in advance by learning using the recognition results when the person is correctly recognized as training data. This learning model is installed in a server device, and inference processing is performed using the learned model with a partially extracted video as input, and the recognition result can be obtained as an output.

[0077] As described above, a trained model for inputting a video of a portion where a face is detected and a trained model for inputting a video of a portion where a movement is detected may be separately prepared. In this case, the video of a portion where a face is detected among the videos is used as input data, and a recognition result when a person is correctly recognized from the video of the portion where a face is detected is learned as training data, and a trained model is generated in advance. Similarly, a trained model for inputting a video of a portion where a movement is detected is used as input data, and a recognition result when a person is correctly recognized from the video of the portion where a movement is detected is learned as training data, and a trained model is generated in advance. In this way, the subject can be recognized more accurately.

[0078] When implementing a learning model such as deep learning, a GPU may be used in addition to a CPU since a larger amount of data needs to be processed in parallel. This allows for more efficient processing. Specifically, when executing a learning program including a learning model, the CPU and GPU work together to perform calculations to learn. The calculations for the learning process may be performed only by the GPU. Similarly, the GPU may also be used for inference processing.

[0079] In the above description, the processing is performed using a trained model in which the object recognition function has been machine-learned. However, rule-based processing such as a look-up table (LUT) may also be performed. In that case, for example, the relationship between the input data and output data of the trained model is created in advance as an LUT. Then, it is advisable to store this created LUT in the memory of the device. When performing object recognition processing, the output data can be obtained by referring to this stored LUT. In other words, the LUT performs object recognition processing by operating in cooperation with a CPU or GPU as a program for performing processing equivalent to the trained model.

[0080] Furthermore, in the above embodiment, a configuration in which tags are uniformly assigned to the entire video has been taken as an example, but a configuration in which tags are assigned in association with timestamps may also be used.

[0081] In the above embodiment, the subject recognition is performed after the video shooting is completed, but the subject recognition may be performed during video shooting. That is, when a face or movement is detected during video shooting, a video of several frames including the detected frame is generated, and the subject recognition may be performed.

[0082] (Other embodiments) The present invention can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having a computer in the system or device read and execute the program. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0083] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gateway (FPGA), a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0084] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0085] 100: information processing device, 101: control unit (extraction means), 102: memory, 103: non-volatile memory, 104: operation unit, 105: display unit, 106: storage medium, 107: imaging unit, 110: communication unit

Claims

1. An acquisition means for acquiring a first moving image; an extracting means for extracting a part of an image from the first moving image; A recognition unit that performs a recognition process to recognize a subject included in the first moving image; a determination means for determining whether or not a processing load required for the recognition process should be reduced; a control means for controlling the recognition means to perform the recognition process using the part of the image extracted from the first video when the determination means determines that a processing load required for the recognition process should be reduced, and for controlling the recognition means to perform the recognition process using the first video when the determination means determines that a processing load required for the recognition process should not be reduced; Equipped with The information processing apparatus is characterized in that the determination means determines that a processing load required for the recognition processing should be reduced when an operation that utilizes a result of the recognition processing is accepted.

2. When a processing load required for the recognition process should be reduced, the control means controls the extraction means to extract a portion of an image from the first video sequence, and then controls the recognition means to perform the recognition process using the portion of the image extracted from the first video sequence; The information processing device described in claim 1, characterized in that when the processing load required for the recognition processing needs not to be reduced, the control means controls the recognition means to perform the recognition processing using the first moving image without causing the extraction means to extract a portion of an image from the first moving image.

3. 3. The information processing apparatus according to claim 1, wherein the part of the images is a moving image or a still image.

4. 4. The information processing apparatus according to claim 1, wherein a processing load required for the recognition process using the part of the images is smaller than a processing load required for the recognition process using the first moving image.

5. 5. The information processing apparatus according to claim 1, wherein the recognition means transmits an image used in the recognition process to an external server, and receives a result of the recognition process from the server.

6. 6. The information processing apparatus according to claim 1, wherein the determining means determines that the processing load required for the recognition process should be reduced when the remaining battery charge is equal to or less than a predetermined value.

7. 7. The information processing device according to claim 1, wherein the determination means determines that a processing load required for the recognition process should be reduced when a cumulative communication volume of the information processing device is equal to or greater than a predetermined volume.

8. 8. The information processing apparatus according to claim 1, wherein the part of the images is an image including a portion of the first video image where a face or movement is detected.

9. 9. The information processing apparatus according to claim 1, further comprising a storage unit that stores the result of the recognition process obtained by the recognition unit in association with the first moving image.

10. The information processing device according to any one of claims 1 to 9, characterized in that after the control means has caused the recognition means to perform the recognition process using the part of the image, if it is determined that it is no longer necessary to reduce the processing load required for the recognition process, the control means controls the recognition means to perform the recognition process using the first moving image.

11. 11. The information processing apparatus according to claim 1, wherein the acquisition means is an image capturing means for capturing an image of a subject.

12. an acquisition step of acquiring a first moving image; an extraction step of extracting a portion of an image from the first video sequence; a recognition step of performing a recognition process of recognizing a subject included in the first moving image; a determination step of determining whether or not a processing load required for the recognition process should be reduced; a control step of controlling, in the recognition step, to perform the recognition process using the part of the image extracted from the first video when it is determined in the determination step that the processing load required for the recognition process should be reduced, and, in the recognition step, to perform the recognition process using the first video when it is determined in the determination step that the processing load required for the recognition process should not be reduced; having The information processing method according to the present invention, wherein, in the determination step, when an operation that utilizes a result of the recognition process is accepted, it is determined that a processing load required for the recognition process should be reduced.

13. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 11.

14. 12. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the information processing device according to claim 1.

Images