Information processing system, information processing device, information processing method, article manufacturing method, program, and recording medium

Abstract

To reduce a load on machine resources.SOLUTION: An information processing system includes: a first imaging unit; a second imaging unit that is arranged at a location different from the first imaging unit; and an information processing unit. The information processing unit performs: obtaining a score indicating an imaged state of a predetermined part of a person who appears in a first captured image which is obtained by imaging by the first imaging unit; and executing either a first operation of recording a second captured image which is obtained by imaging by the second imaging unit or a second operation of not recording the second captured image, based on the score.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a technique for detecting a person. [Background technology]

[0002] In various production sites such as factories and manufacturing facilities, when people are engaged in the production of goods, not only the performance of the production equipment but also the movements of the people affect productivity. Patent Document 1 discloses a method of capturing an image of a person with two cameras and estimating the posture of the person based on the captured images obtained from the two cameras. Patent Document 2 discloses a method of capturing an image of a person with multiple cameras and detecting the body parts of the person. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-97577 A [Patent Document 2] JP 2017-59945 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned method tends to place a heavy load on machine resources.

[0005] An object of the present invention is to reduce the load on machine resources. [Means for solving the problem]

[0006] According to a first aspect of the present invention, an information processing system includes a first imaging unit, a second imaging unit arranged at a position different from the first imaging unit, and an information processing unit, wherein the information processing unit obtains a score indicating an imaging state of a predetermined part of a person appearing in a first captured image obtained by imaging of the first imaging unit, and executes a first operation of recording a second captured image obtained by imaging of the second imaging unit or a second operation of not recording the second captured image based on the score.

[0007] According to a second aspect of the present invention, there is provided an information processing device including an information processing unit, wherein the information processing unit obtains a score indicating an imaging state of a predetermined part of a person appearing in a first captured image obtained by imaging with a first imaging unit, and executes a first operation of recording a second captured image obtained by imaging with a second imaging unit, or a second operation of not recording the second captured image, based on the score.

[0008] According to a third aspect of the present invention, an information processing method includes obtaining a score indicating an imaging state of a specific part of a person appearing in a first captured image obtained by imaging with a first imaging unit, and, based on the score, executing a first operation of recording a second captured image obtained by imaging with a second imaging unit, or a second operation of not recording the second captured image. Effect of the Invention

[0009] According to the present invention, the load on machine resources is reduced. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram of an information processing system according to an embodiment. [Diagram 2] 1 is a block diagram showing an information processing system according to an embodiment. [Diagram 3] FIG. 2 is a functional block diagram showing functions of the information processing system according to the embodiment. [Figure 4] 1 is a flowchart of an information processing method according to an embodiment. [Diagram 5]1 is a flowchart of an information processing method according to an embodiment. [Figure 6] FIG. 13 is a diagram illustrating an example of an output of a blind spot level score according to the embodiment. [Figure 7] FIG. 11 is an explanatory diagram illustrating an example of an analysis process according to the embodiment. [Figure 8] 3A and 3B are schematic diagrams of a user interface image according to the embodiment. [Figure 9] 3A and 3B are schematic diagrams of a user interface image according to the embodiment. [Figure 10] 3A and 3B are schematic diagrams of a user interface image according to the embodiment. [Figure 11] 3A and 3B are schematic diagrams of a user interface image according to the embodiment. [Figure 12] 3A and 3B are schematic diagrams of a user interface image according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Fig. 1 is an explanatory diagram of an information processing system 100 according to an embodiment. Fig. 2 is a block diagram showing the information processing system 100 according to the embodiment.

[0012] In this embodiment, the information processing system 100 is a system for analyzing the behavior of a person P1 who is an analysis target. The person P1 is, for example, a worker engaged in the manufacture of goods at a production site such as a factory. For example, around the person P1, workpieces used in the manufacture of goods or boxes containing the workpieces are arranged. Also, for example, around the person P1, production equipment is arranged. That is, the person P1 may work around the production equipment installed at a predetermined position.

[0013] The information processing system 100 includes an information processing device 300 and cameras 201 and 202 as a plurality of imaging units. The camera 201 is an example of a first imaging unit. The camera 202 is an example of a second imaging unit. In this embodiment, the camera 201 is a main camera, and the camera 202 is a sub-camera.

[0014] The cameras 201 and 202 and the information processing device 300 are connected to each other via a network NW so as to be able to communicate with each other via a PoE (Power over Ethernet) hub 101 and a router 102. By using the PoE hub 101, it is possible to supply power to the cameras 201 and 202.

[0015] Each of the cameras 201 and 202 is a digital camera and has an image sensor (not shown). The image sensor is, for example, a CMOS image sensor or a CCD image sensor. Each of the cameras 201 and 202 is, for example, a network camera and is controlled by the information processing device 300 via the network NW.

[0016] Each of the cameras 201 and 202 is disposed at a position where the person P1 can be imaged. Each of the cameras 201 and 202 is disposed at a position where the entire body of the person P1 can be included in the imaging range of each of the cameras 201 and 202 when there is no obstruction. The imaging range is an angle of view. The cameras 201 and 202 are disposed in a predetermined space R0, for example, a work space where the person P1 works. In the predetermined space R0, the camera 202 is disposed at a different position from the camera 201, and can image the person P1 in the predetermined space R0 from a different direction from the camera 201. For example, the cameras 201 and 202 are disposed at different positions from each other in the predetermined space R0 such that the camera 201 images the person P1 from the front and the camera 202 images the person P1 from the back.

[0017] Here, the person P1 includes clothing. The person P1 has a plurality of body parts, such as a head, a waist, an elbow, and a knee. When a body part of the person P1 is covered by an object W1, some or all of the body parts of the person P1 may not be captured in the captured image captured by the camera 201. In the following description, when a body part of the person P1 is covered by an object W1 in the imaging range of the camera 201 and is not captured in the captured image captured by the camera 201, it is said that a blind spot occurs for the person P1. In the following description, a body part of the person P1 that is covered by a shield and is not captured in the captured image captured by the camera 201 is also called a blind spot. The camera 201 is installed in a position where a blind spot does not occur for the person P1 as much as possible. In addition, the camera 202 is installed in a position where a blind spot can be captured when a blind spot occurs for the person P1 captured by the camera 201.

[0018] The object W1 may be any object, but the person P1 works on a production line in a factory or the like. Therefore, the object W1 may be production equipment, or may be a work or a box containing a work. The work is, for example, a part used to manufacture an article.

[0019] The camera 201 can capture a person P1 in a moving image M1. The camera 202 can also capture a person P1 in a moving image M2. The moving image M1 is an example of a first moving image. The moving image M2 is an example of a second moving image. The moving image M1 is a bundle of a plurality of captured images I1 generated at a predetermined frame rate. The moving image M2 is also a bundle of a plurality of captured images I2 generated at a predetermined frame rate. Each captured image I1 and each captured image I2 are also called a frame. For example, when the predetermined frame rate is 30 fps, the moving image M1 per minute can be said to be a bundle of captured images I1 of 1800 frames. The captured image I1 is an example of a first captured image. The captured image I2 is an example of a second captured image. Hereinafter, the captured image I1 may also be called a frame I1. The captured image I2 may also be called a frame I2.

[0020] The information processing device 300 is configured with a computer. The information processing device 300 can send an image capture start command to the cameras 201, 202 to cause the cameras 201, 202 to start capturing images, and can send an image capture stop command to the cameras 201, 202 to cause the cameras 201, 202 to stop capturing images. The information processing device 300 is configured to be able to acquire the video M1 generated by the camera 201 and the video M2 generated by the camera 202 via the network NW. The information processing device 300 is also configured to be able to record the acquired videos M1, M2. The information processing device 300 is also configured to be able to process the recorded videos M1, M2.

[0021] The information processing device 300 includes a central processing unit (CPU) 351, which is an example of a processor. The CPU 351 functions as an information processing unit by executing a program 361. The information processing device 300 also includes a read only memory (ROM) 352, a random access memory (RAM) 353, and a solid state drive (SSD) 354 as storage units. The information processing device 300 also includes an I / O 355, which is an input / output interface. The information processing device 300 also includes a display 356, which is an example of a display unit, and a keyboard 358 and a mouse 357, which are examples of an input unit. The CPU 351, the ROM 352, the RAM 353, the SSD 354, the I / O 355, the display 356, the keyboard 358, and the mouse 357 are connected to each other via a bus so as to be able to perform data communication with each other.

[0022] The ROM 352 stores basic programs related to the operation of the computer. The RAM 353 is a storage device that temporarily stores various data such as the results of calculations performed by the CPU 351. The SSD 354 records the results of calculations performed by the CPU 351 and various data acquired from the outside, and also records a program 361 for causing the CPU 351 to execute various processes. The program 361 is application software that can be executed by the CPU 351.

[0023] The CPU 351 can execute information processing and control processing, which will be described later, by executing the program 361 recorded in the SSD 354. For example, the CPU 351 can execute the program 361 to control the cameras 201 and 202 via the network NW and acquire captured images I1 and I2, which are image data, that is, moving images M1 and M2, from the cameras 201 and 202 via the network NW.

[0024] The I / O 355 is an interface with an external device, and is connected to the network NW for data communication, for example, by being connected wirelessly or by wire to the router 102. In addition, a removable memory 362 such as a recording disk or a memory device can be connected to the I / O 355, and the I / O 355 can read various data, programs, and the like recorded in the removable memory 362.

[0025] The information processing apparatus 300 includes an SSD 354 as a storage device, but is not limited to this. The storage device, or storage, included in the information processing apparatus 300 may be, for example, an HDD.

[0026] In the present embodiment, the non-transitory computer-readable recording medium is the SSD 354, and the program 361 is recorded in the SSD 354, but this is not limiting. The program 361 may be recorded in any recording medium as long as it is a non-transitory computer-readable recording medium. As a recording medium for supplying the program 361 to the computer, a removable memory 362, such as a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, a non-volatile memory, or the like, can be used.

[0027] 3 is a functional block diagram showing functions of the information processing system 100 according to the embodiment. A CPU 351 of the information processing device 300 functions as an information processing unit 301 by executing a program 361.

[0028] The information processing unit 301 includes a function of acquiring captured images I1, which are frames constituting a moving image M1, from the camera 201, recording them in the SSD 354, and executing image processing of the captured images I1. The information processing unit 301 also includes a function of acquiring captured images I2, which are frames constituting a moving image M2, from the camera 202, recording them in the SSD 354, and executing image processing of the captured images I2. The information processing unit 301 also includes a function of executing control of the cameras 201, 202 and control of the display 356. The information processing unit 301 also includes a function of executing analysis processing to analyze the movements of the person P1.

[0029] The moving images M1 and M2 obtained by imaging are recorded in a storage device of the information processing device 300, for example, the SSD 354. Note that the destination for recording the moving images M1 and M2 is not limited to the SSD 354, and may be, for example, the removable memory 362 or a memory device built into or externally attached to the cameras 201 and 202. Also, the moving images M1 and M2 may be a storage device, i.e., a storage, connected to the network NW.

[0030] When the program 361 is started in the information processing device 300, the CPU 351 functions as the information processing unit 301. The information processing unit 301 is configured to be switchable between a test mode and an operation mode. The operation mode includes a recording mode and an analysis mode.

[0031] Fig. 4 and Fig. 5 are flowcharts of the information processing method according to the embodiment. Fig. 4 is a flowchart showing the processing operation of the information processing unit 301 when the recording mode is set, and Fig. 5 is a flowchart showing the processing operation of the information processing unit 301 when the analysis mode is set.

[0032] The information processing unit 301 can selectively execute a recording mode shown in Fig. 4 or an analysis mode shown in Fig. 5. The recording mode is a mode for recording a moving image of a person P1. The analysis mode is a mode for analyzing the movement of the person P1 based on the moving image. Note that a part of the processing of the analysis mode may be performed in the recording mode. The test mode is a mode for setting a threshold value, and will be described in detail later.

[0033] First, the process of the information processing unit 301 in the recording mode will be described. In step S101, the information processing unit 301 activates the cameras 201 and 202. This causes the cameras 201 and 202 to capture moving images. Here, the frame rate of the moving images captured by the camera 201 is the same as the frame rate of the moving images captured by the camera 202. The imaging timing of the camera 201 is synchronized with the imaging timing of the camera 202.

[0034] Next, in step S102, the information processing unit 301 starts recording the video M1. Here, the camera 202 is in an activated state, and the video M2 is generated by the image pickup of the camera 202, but the recording of the video M2 has not started. An operation of the information processing unit 301 not recording the captured image I2, i.e., an operation not recording the video M2, is called a sleep operation. On the other hand, an operation of the information processing unit 301 recording the captured image I2, i.e., an operation recording the video M2, is called a recording operation. The recording operation is an example of a first operation, and the sleep operation is an example of a second operation. The camera 201 can also be switched between a recording operation and a sleep operation, but in this embodiment, the information processing unit 301 executes a recording operation on the camera 201 in steps S102 to S107.

[0035] Next, in step S103, the information processing unit 301 records one frame of the captured image I1 in the moving image M1, and calculates part information corresponding to the part of the person P1 and a score corresponding to the part based on the recorded one frame of the captured image I1.

[0036] The score corresponding to a part is a score indicating the imaging state of the part of person P1 captured in captured image I1 obtained by imaging with camera 201. For example, the score is a score indicating the degree of obscuration of the part, which indicates the extent to which the part of person P1 captured in captured image I1 is obscured. Hereinafter, the score is referred to as a blind spot level score. Note that the imaging state of the part may be, other than the degree of obscuration of the part, the visibility of the part, which indicates the extent to which the part is visible.

[0037] The part information is information about the positions of the parts of the person P1, and in this embodiment, is skeletal coordinate information. The skeletal coordinate information is acquired by a technique for estimating a posture from a still image or a moving image using a skeletal model of a person. For example, it is acquired by a known technique for obtaining skeletal coordinate information, such as OpenPose or MediaPipe. The part information may be acquired by a segmentation technique for grouping pixels in the captured image I1 into meaningful regions. The skeletal coordinate information is acquired, for example, as three-dimensional coordinates in meters with the center of the waist as the origin, or, for example, as two-dimensional coordinates with pixel information as the unit when the corner of the image is the origin.

[0038] The following description focuses on a specific part, which is one part of the person P1. The blind spot level score of the specific part is calculated when acquiring part information of the specific part. The information processing unit 301 estimates the position of the specific part in the captured image I1 around a key point part such as the head, and calculates the accuracy of the estimated position as the blind spot level score. Note that the method of calculating the blind spot level score is not limited to this. For example, the information processing unit 301 may estimate the position of the specific part of the person P1 from the detected position of the person P1, and calculate the amount by which the specific part is hidden based on the pixels in the captured image I1 relative to the estimated position.

[0039] The information processing unit 301 obtains the part information and the blind spot level score of each part of the person P1 every time one frame of the captured image I1 is obtained, thereby obtaining the blind spot level score in almost real time.

[0040] Fig. 6 is a diagram showing an example of output of blind spot level scores according to the embodiment. Fig. 6 focuses on the "left knee" as a predetermined part of person P1, and illustrates the blind spot level score of the predetermined part when person P1 is captured in video M1 by camera 201. The horizontal axis of the graph shown in Fig. 6 indicates the frame number of video M1, and the vertical axis of the graph shown in Fig. 6 indicates the blind spot level score of the predetermined part corresponding to the frame of the frame number. In video M1, frame numbers are assigned in chronological order.

[0041] The blind spot level score is output as a numerical value between 0 and 1. As described above, the blind spot level score indicates the accuracy of the estimated position of the estimated specific part. The higher the numerical value indicating the blind spot level score, the lower the accuracy of the estimated position of the estimated specific part. The calculation and accuracy of the estimated position are obtained, for example, using a trained model obtained by machine learning.

[0042] In frame I11, the blind spot level score indicates a value close to 0, and the skeletal coordinate position of the left knee is accurately estimated. On the other hand, in frame I12, the blind spot level score indicates a value close to 1, and the skeletal coordinate position 503 of the left knee is estimated to be shifted from the actual position 504 of the left knee.

[0043] In step S104, the information processing unit 301 judges the value of the blind spot level score acquired in step S102. In this embodiment, to judge the value of the blind spot level score, the information processing unit 301 judges whether the blind spot level score exceeds a threshold TH1 or is equal to or less than the threshold TH1. The threshold TH1 is an example of a predetermined value. Moreover, a range R1 of 0 or more and equal to or less than the threshold TH1 is an example of a predetermined range. When the blind spot level score exceeds the threshold TH1, that is, when the blind spot level score falls outside the range R1, it indicates a state in which a blind spot is present. When the blind spot level score is equal to or less than the threshold TH1, that is, when the blind spot level score is included in the range R1, it indicates a state in which no blind spot is present.

[0044] If step S104 is YES, that is, if the information processing unit 301 determines that the blind spot level score is equal to or lower than the threshold value TH1, in other words, if the information processing unit 301 determines that the blind spot level score is within the range R1, the information processing unit 301 proceeds to the process of step S105.

[0045] If step S104 is NO, that is, if the information processing unit 301 determines that the blind spot level score exceeds the threshold value TH1, in other words, if the information processing unit 301 determines that the blind spot level score is outside the range R1, it proceeds to processing of step S106.

[0046] In step S105, the information processing unit 301 executes a sleep operation and does not record the video M2. That is, if the information processing unit 301 was recording the video M2, it stops recording the video M2. If the information processing unit 301 was not recording the video M2, it continues to stop recording the video M2.

[0047] In step S106, the information processing unit 301 executes a recording operation to record the video M2. That is, if the information processing unit 301 has been recording the video M2, the information processing unit 301 continues recording the video M2. Also, if the information processing unit 301 has not been recording the video M2, the information processing unit 301 starts recording the video M2.

[0048] When starting recording of the video M2, the information processing unit 301 links the first frame I2 in the video M2 with the frame I1 of the video M1 captured at the same time. For example, the first frame I2 of the video M2 is assigned a frame number that is the same as the frame number of the frame I1 of the video M1 captured at the same time as the first frame I2. Subsequent frames I2 in the video M2 are also assigned in a similar manner. In this way, each frame I2 included in the video M2 is linked with the corresponding frame I1 of the video M1. Note that the frame I2 of the video M2 is linked with the frame I1 of the video M1 captured at a later time than the frame I1 of the video M1 used in the determination of step S104.

[0049] When the blind spot level score exceeds the threshold TH1, the operation of recording the video M2 captured by the camera 202 continues until the blind spot level score becomes equal to or less than the threshold TH1. At this time, a threshold TH2 different from the threshold TH1 may be used instead of the threshold TH1. In other words, the threshold used for determining when to switch from the sleep operation to the recording operation may be the threshold TH1, and the threshold used for determining when to switch from the recording operation to the sleep operation may be the threshold TH2.

[0050] While the determination of the left knee has been described as an example above, the information processing unit 301 performs the same determination as the left knee for all parts of the person P1.

[0051] In step S107, the information processing section 301 determines whether or not the user has operated the process of stopping the recording mode.

[0052] If the determination in step S107 is NO, i.e., if the stop process has not been operated, the information processing unit 301 returns to the process of step S103, and in step S103, performs the same process on the frame I1 of the video M1 to be recorded next as on the previous frame I1. In this way, in the routine that repeats steps S103 to S107, the information processing unit 301 calculates and obtains a blind spot level score every time it obtains a captured image I1 in the video M1, i.e., frame I1. Thereby, the process of obtaining the blind spot level score is performed almost in real time, and it is determined whether or not to record the video M2.

[0053] If the determination in step S107 is YES, that is, if the stop process has been operated, the information processing unit 301 proceeds to the next process in step S108.

[0054] In step S108, the information processing unit 301 stops imaging from both cameras 201 and 202. The information processing unit 301 also stops recording of the video M1, and if video M2 is being recorded, also stops recording of the video M2.

[0055] As described above, the information processing unit 301 selectively executes a recording operation for recording the captured image I2 captured by the camera 202, or a sleep operation for not recording the captured image I2, based on the blind spot level score. In this way, the information processing unit 301 preferably constantly records the video M1 in the recording mode, and preferably switches the recording of the video M2 on / off based on the blind spot level score. In this way, by switching the recording of the video M2 on / off, the load on the machine resources, for example, storage such as the SSD 354, can be reduced. Reducing the load on the SSD 354 means reducing the amount of storage area used in the SSD 354. In other words, it is possible to ensure sufficient free space in the SSD 354, and to handle long-term recording.

[0056] Next, the processing of the information processing unit 301 in the analysis mode, that is, the analysis processing, will be described. The information processing unit 301 starts the analysis processing according to the flow shown in Fig. 5. In this embodiment, the analysis is performed in order to discover the load on the knees or waist of the person P1, for example, or an inefficient movement of the person P1. For example, the person P1 refers to the analysis result to improve the efficiency of the movement when working on a production line or the like to manufacture goods, thereby improving the productivity of the goods.

[0057] The following analysis process is performed frame by frame in chronological order. First, in step S109, the information processing unit 301 sets one frame I1 of the video M1 as the target frame I1, and determines whether there is a frame I2 of the video M2 linked to the target frame I1. That is, the information processing unit 301 determines whether there is a frame I2 of the video M2 that corresponds to the target frame I1. In this embodiment, the information processing unit 301 manages by frame number, and determines whether there is a frame I2 with the same frame number as the frame number assigned to the target frame I1.

[0058] If the determination in step S109 is NO, i.e., if there is no frame I2 in the moving image M2 corresponding to the target frame I1, the information processing unit 301 proceeds to the process of step S110. In step S110, the information processing unit 301 analyzes the movement of the person P1 using the skeleton coordinate information, which is the body part information calculated in step S103 based on the target frame I1 of the moving image M1.

[0059] Note that frame I2 of video M2 is linked to frame I1 of video M1 acquired at a later timing than frame I1 of video M1 used in the determination of step S104. Therefore, even if there is no frame I2 of video M2 corresponding to target frame I1 of video M1, the blind spot level score calculated based on target frame I1 may exceed threshold TH1. In other words, a blind spot may occur in person P1 in target frame I1. However, the number of such frames is about one frame or several frames, and has almost no effect on the analysis of the movement of person P1. Also, by adjusting threshold TH1 in advance, analysis errors can be reduced.

[0060] If the determination in step S109 is YES, that is, if there is a frame I2 of the moving image M2 corresponding to the target frame I1, the information processing unit 301 proceeds to the process of step S111. In step S111, the information processing unit 301 calculates skeletal coordinate information, which is part information of each part of the person P1, based on the frame I2 of the moving image M2 corresponding to the target frame I1.

[0061] Here, for the target frame I1 of the video M1, the part information and the blind spot level score for each part of the person P1 have already been calculated in step S103. That is, the blind spot parts in the target frame I1 have already been calculated. In step S112, the information processing unit 301 analyzes the movement of the person P1 using the skeletal coordinate information calculated in step S103 for parts other than the blind spot parts, and using the skeletal coordinate information calculated in step S111 for the blind spot parts. That is, the information processing unit 301 analyzes the movement of the person P1 based on the captured image I1, which is the target frame I1, and the captured image I2, which is the frame I2 linked to the target frame I1.

[0062] Note that frame I2 of video M2 is linked to frame I1 of video M1 acquired at a later timing than frame I1 of video M1 used in the determination of step S104. Therefore, there may be cases where the blind spot level score is equal to or less than threshold value TH1 for each piece of skeletal coordinate information calculated from target frame I1 of video M1. In such a case, there is no blind spot. Therefore, in analyzing the movement of person P1, it is preferable that information processing unit 301 uses skeletal coordinate information based on target frame I1 of video M1, but may also use skeletal coordinate information based on frame I2 of video M2 corresponding to target frame I1.

[0063] That is, the information processing unit 301 analyzes the movement of the person P1 using skeletal coordinate information based on the target frame I1 of the video M1 and / or skeletal coordinate information based on the frame I2 of the video M2 linked to the target frame I1.

[0064] In summary, the information processing section 301 analyzes the movement of the person P1 based on at least one of the captured image I1 and the captured image I2.

[0065] 7 is an explanatory diagram showing an example of the analysis process according to the embodiment. A moving image M1 includes, for example, a frame I1 showing a captured image. A ,I1 B ,I1 CThe moving image M2 includes, for example, a frame I2 showing a captured image. B Frame I1 B is frame I2 of video M2. B It is linked to Frame I1 A ,I1 C is not associated with any frame of the video M2.

[0066] Frame I1 A There is no corresponding frame in the video M2 for frame I1. A The movement of the person P1 is analyzed using each skeleton coordinate information A1 based on the frame I1. C There is no corresponding frame in the video M2 for frame I1. C The movement of the person P1 is analyzed using each piece of skeletal coordinate information A1 based on the above.

[0067] On the other hand, frame I1 B For frame I2 B The information processing unit 301 corresponds to frame I2 B Here, the blind spot level score for the skeletal coordinate information A11 corresponding to the right knee and the blind spot level score for the skeletal coordinate information A12 corresponding to the right ankle are both greater than the threshold TH1. Therefore, it is highly likely that the person's right knee and right ankle are hidden by an obstruction. Therefore, the information processing unit 301 calculates the blind spot level score for the skeletal coordinate information A11 corresponding to the right knee and the blind spot level score for the skeletal coordinate information A12 corresponding to the right ankle from the frame I2. B Among the skeletal coordinate information A2 obtained based on the above, skeletal coordinate information A21 corresponding to the right knee and skeletal coordinate information A22 corresponding to the right ankle are extracted, and the motion of the person P1 is analyzed using the skeletal coordinate information A1 other than the skeletal coordinate information A11 and A12 and the skeletal coordinate information A21 and A22. In other words, the information processing unit 301 replaces the skeletal coordinate information A11 and A12 with the skeletal coordinate information A21 and A22 among the multiple skeletal coordinate information A1, and analyzes the motion of the person P1.

[0068] The following description focuses on the right knee as the predetermined part. AThe information processing unit 301 also acquires the skeleton coordinate information A11 based on the frame I2 B The information processing unit 301 also acquires the skeleton coordinate information A21 based on the frame I1 C In this way, the information processing unit 301 acquires skeletal coordinate information of the right knee based on the captured image I1 or I2. The information processing unit 301 acquires skeletal coordinate information of other parts in the same manner as the left knee.

[0069] The analysis result of the movement of the person P1 acquired in step S110 or step S111 includes information about the posture of the person P1. The information about the posture of the person P1 is, for example, the angle of each joint of the person P1 and the movement distance of each part between frames.

[0070] In step S113, the information processing unit 301 outputs the analysis result to, for example, the display 356, thereby displaying information indicating the analysis result on the display 356.

[0071] By referring to the analysis results, the user can recognize the load on the knees and hips of the person P1 and inefficient movements. The user may be the person P1 or a person other than the person P1. When the user is the person P1, the person P1 can perform efficient movements by referring to the analysis results, thereby improving productivity. When the user is a person other than the person P1, the user can simply instruct the person P1 to perform efficient movements.

[0072] Fig. 8 is a schematic diagram of a user interface image (UI image) UI1 according to the embodiment. Fig. 9 is a schematic diagram of a UI image UI2 according to the embodiment. The UI images UI1 and UI2 shown in Fig. 8 and Fig. 9 are examples of UI images for displaying analysis results.

[0073] As shown in Fig. 8, the information processing unit 301 displays a UI image UI1 on the display 356 of Figs. 2 and 3. The UI image UI1 includes a plurality of selection items C1 to C7 that the user can select using a pointing device such as a mouse 357. For example, seven selection items C1 to C7 corresponding to seven body parts are displayed so as to be selectable. Each selection item is associated with information on the analysis result of each body part.

[0074] The information processing unit 301 displays information on the analysis result of the body part corresponding to the selected selection item from among the multiple selection items C1 to C7 in a UI image UI2 shown in FIG. 9. The UI image UI2 includes a display area R11 for playing the video M1 and a display area R12 for displaying a graph showing the analysis result as an image. In the display area R11, an image E1 of a skeleton model corresponding to the skeleton coordinate information A1 or A2 is displayed as an animation superimposed on the video M1. In the display area R11, a button B1 for playing the video M1 is displayed. When the user selects the button B1 using a pointing device such as a mouse 357, the information processing unit 301 plays and displays the video M1 in the display area R11.

[0075] In the analysis results, the angles of the joints and the moving distances of the parts between frames are acquired as time-series data. Therefore, the image of the graph displayed in the display area R12 is visualized so that the time series can be understood. Note that FIG. 9 illustrates a case where all the selection items C1 to C7 in FIG. 8 are selected. Image D1 is an image of the analysis result corresponding to the selection item C1. Image D1 shows the analysis result of the right hand movement. Image D2 is an image of the analysis result corresponding to the selection item C2. Image D2 shows the analysis result of the left hand movement. Images D31 and D32 are images of the analysis result corresponding to the selection item C3. Image D31 shows the analysis result of the twisting angle of the waist. Image D32 shows the analysis result of the bending angle of the waist. Image D4 is an image of the analysis result corresponding to the selection item C4. Image D4 shows the joint angle of the right knee. Image D5 is an image of the analysis result corresponding to the selection item C5. Image D5 shows the joint angle of the right ankle. Image D6 is an image of the analysis result corresponding to selection item C6. Image D6 shows the joint angle of the left knee. Image D7 is an image of the analysis result corresponding to selection item C7. Image D7 shows the joint angle of the left ankle.

[0076] 9, the shaded portions indicate the analysis results obtained in step S112 based on frame I2 of moving image M2. The remaining portions indicate the analysis results obtained in step S111 based on frame I1 of moving image M1. In this way, when it is determined that a blind spot has occurred in a body part in moving image M1, the movement of person P1 is analyzed by supplementing it with moving image M2.

[0077] Here, during the processing of steps S101 to S108, i.e., in the operation mode, the information processing unit 301 displays the usage status of machine resources on the display 356. Figures 10 and 11 are schematic diagrams of a UI image UI3 according to the embodiment. Figure 10 illustrates a UI image UI3 in a state where the information processing unit 301 is performing a recording operation, and Figure 11 illustrates a UI image UI3 in a state where the information processing unit 301 is performing a sleep operation.

[0078] When the information processing device 300, that is, the program 361, is started, the information processing unit 301 displays a UI image UI3 as shown in Fig. 10 on the display 356. The UI image UI3 includes display areas 601-607.

[0079] In the display area 601, the video M1 being recorded is displayed. In the display area 602, the video M2 being recorded is displayed. In the display area 603, the history of blind spots that have occurred is displayed. In the display area 604, the state of the camera 202 is displayed. In the display area 605, the usage amount of the storage in which the video M2 is recorded, for example, the usage rate, is displayed. In the display area 606, the usage amount of the bandwidth of the network NW, for example, the amount of data received per unit time by the information processing device 300 via the network NW, is displayed. In the display area 607, the usage rate of the CPU 351 is displayed. The UI image UI3 is updated almost in real time at a predetermined time interval. This allows the user to check the status of the information processing system 100 almost in real time. In addition, it becomes easy for the operator to adjust the positions of the cameras 201 and 202 while looking at the UI image UI3.

[0080] The video M2 is displayed only when it is being recorded in the display area 602. When the video M2 is not being recorded, the display area 602 displays something other than the video M2, for example a monochrome screen of black or gray.

[0081] In the display area 603, the name of the body part where the blind spot occurred, the number of times the blind spot occurred, and the word "details" are displayed for each body part. When the word "details" is selected, a graph showing the blind spot level score of the corresponding body part in time series is displayed, as shown in Fig. 6. This allows the user to check the blind spot level score for the selected body part.

[0082] In the display area 604, three items are displayed: "Sub camera status," "Blind spot occurrence location," and "Sub camera operation rate."

[0083] In the "Sub camera state" item, the information processing unit 301 displays whether it is performing a recording operation for recording the moving image M2 obtained by imaging the camera 202, or performing a sleep operation in which the information processing unit 301 is not recording the moving image M2. This allows the user to confirm whether it is in a recording operation state or a sleep operation state.

[0084] In the "Blind Spot Location" item, the information processing unit 301 displays the name of the blind spot. If no blind spot exists, the word "none" is displayed. This display allows the user to confirm where the blind spot exists. The information processing unit 301 may display a skeleton model image of a person in the display area 604, and may surround the areas in the skeleton model image where the blind spot exists with a line.

[0085] In the "sub-camera operating rate" field, the information processing unit 301 displays the operating rate of the camera 202, i.e., the ratio of the recording time of the video M2 to the recording time of the video M1, for example, as a percentage. This allows the user to check the operating rate of the camera 202.

[0086] Display areas 605-607 are areas for displaying the usage status of machine resources of information processing system 100. Information processing unit 301 displays the usage status of machine resources in display areas 605-607. Information processing unit 301 displays information related to the recording time of video M2 as the usage status of machine resources in display area 605. Furthermore, information processing unit 301 displays the usage status of network NW as the usage status of machine resources in display area 606. Furthermore, information processing unit 301 displays the usage status of CPU 351 as the usage status of machine resources in display area 607.

[0087] In the display area 605, the usage of the storage in which the video M2 is recorded is displayed, for example, in a time series graph, as information related to the recording time of the video M2. In the recording mode, the video M1 is constantly recorded, but it is assumed that the recording of the video M2 is started and stopped repeatedly. Therefore, by displaying information related to the recording time of the video M2 in the display area 605, the load on the storage can be visually confirmed. Here, the storage is, for example, the SSD 354, as described above. The horizontal axis of the graph represents time, and is updated, for example, every few seconds. It is to be noted that when the video M2 is not being recorded, the usage of the storage does not change over time, as shown in FIG. 11.

[0088] The display area 606 displays the bandwidth usage of the network NW in the recording mode. For example, the bandwidth usage rate of the network NW is visualized as a graph as a load. The horizontal axis of the graph represents time. The graph is updated every few seconds. As shown in FIG. 11, when no blind spot occurs, the video M2 is not being recorded, so the bandwidth usage rate is reduced.

[0089] The display area 607 displays the utilization rate of the CPU 351 of the information processing device 300. In this way, the utilization rate of the CPU 351 is visualized as a load in a graph in the display area 607. Thus, the user can check the utilization status of the machine resources from the UI image UI3. The horizontal axis of the graph represents time. The graph is updated every few seconds. That is, the UI image UI3 is updated almost in real time. Thus, the user can check the utilization status of the machine resources from the UI image UI3 almost in real time. Note that, as shown in FIG. 11, when no blind spot occurs, the video image M2 is not recorded, and therefore the utilization rate of the CPU 351 is reduced.

[0090] Here, when the remaining amount of data writable storage area in the SSD 354, that is, the free space of the SSD 354, reaches a set value, the information processing unit 301 may delete videos recorded in the SSD 354 in order starting from the oldest.

[0091] As described above, according to the embodiment, by providing a period during which the video M2 is not recorded while the video M1 is being recorded, the load on the machine resources of the information processing system 100, such as the CPU 351, the SSD 354, and the devices that configure the network NW, is reduced. The devices that configure the network NW are, for example, the PoE hub 101 and the router 102.

[0092] Incidentally, the threshold value TH1 used in the above-mentioned operation mode is set in a storage device such as the SSD 354. An example of a method for setting the threshold value TH1 will be described. As shown in FIG. 6, a range in which the body part information cannot be estimated with high accuracy in the blind spot level score is set as a range 506. The threshold value TH1 is determined from two values ​​described below so as not to be included in this range 506. The first value is a blind spot level score value 508 that outputs skeleton coordinates that can be determined to be highly reliable when comparing the results of outputting skeleton coordinates and the results of the blind spot level score at that time in multiple test videos. The second value is a delay time 507 of the start of recording of the video image M2 that occurs when switching from a sleep operation to a recording operation. That is, a delay of several frames occurs from when the information processing unit 301 determines to start recording to when recording actually starts.

[0093] Therefore, it is preferable that the setting of the threshold value TH1 be changeable by the user. In this embodiment, the information processing unit 301 accepts a change in the range R1, i.e., the threshold value TH1, in the above-mentioned test mode, performs a computer simulation both before and after the change in the threshold value TH1, and compares the simulation results and displays them on the display 356.

[0094] 12 is a schematic diagram of a UI image UI4 according to the embodiment. The information processing unit 301 displays the UI image UI4 for accepting the setting of the range R1, i.e., the threshold value TH1, on the display 356. The UI image UI4 is a GUI for the user to set the threshold value TH1.

[0095] The information processing unit 301 displays the result of the computer simulation in the UI image UI4. Hereinafter, the computer simulation will be simply referred to as a simulation. The simulation is also an estimation process.

[0096] First, the information processing unit 301 records, as data to be used in a simulation, moving images M1 and M2 of a person P1 captured by the cameras 201 and 202 for a predetermined time in the SSD 354. During a simulation, the information processing unit 301 reads out the moving images M1 and M2 from the SSD 354. The lengths of the moving images M1 and M2 acquired in the test mode are the same, for example, 10 minutes.

[0097] The SSD 354 stores data of the threshold value TH1 before the change, and the information processing unit 301 reads the data of the threshold value TH1 from the SSD 354 during a simulation.

[0098] The UI image UI4 includes display areas 801 to 810. The information processing unit 301 displays information on the recording time of the video M1 obtained by imaging the camera 201 in the display area 801. The information processing unit 301 also displays a simulation result when the threshold value TH1 before the change is used in the display area 801. Specifically, the information processing unit 301 calculates an estimated time for the blind spot level score to exceed the threshold value TH1 before the change, and displays the estimated time for the recording time of the video M1 in the display area 801 as the operation rate of the camera 202. The information processing unit 301 also calculates an average value of the utilization rate of the CPU 351, an average value of the bandwidth utilization amount of the network NW, and a total power utilization amount for the recording time of the video M1, and displays them in the display area 801.

[0099] The threshold value TH1 can be set for each body part. Therefore, in this embodiment, the information processing unit 301 displays tabs for receiving the selection of a body part in the display area 802. In the example of Fig. 12, "right hip" is selected as the body part.

[0100] The information processing unit 301 calculates the blind spot level score of the part corresponding to the "right hip" in each frame constituting the video M1, and calculates the total time (blind spot time) during which the blind spot level score exceeds the pre-change threshold TH1. The information processing unit 301 also calculates the number of times that a blind spot occurs for the part corresponding to the "right hip".

[0101] The information processing unit 301 displays each piece of information, namely, the total blind spot time, the number of times a blind spot occurred, and the threshold value before change TH1, in the display area 803. The information processing unit 301 also displays an image showing the calculated blind spot level score in a graph in the display area 804. For example, a graph showing the blind spot level score of a selected part in time series, as shown in Fig. 6, is displayed in the display area 804. The information processing unit 301 may display a dotted bar indicating the threshold value before change TH1 on this graph.

[0102] The information processing unit 301 accepts a change to the threshold value TH1 made by the user via the UI image UI4. In this embodiment, a slide bar 806 indicating the changed threshold value TH1 is displayed on the graph displayed in the display area 804.

[0103] The information processing unit 301 accepts the operation of the slide bar 806 by the user, and sets a value according to the position of the slide bar 806 as the changed threshold value TH1. At that time, the information processing unit 301 displays the numerical value of the changed threshold value TH1 in the display area 805. The display area 805 is a box that displays the numerical value indicating the threshold value TH1.

[0104] In this way, the information processing unit 301 changes the threshold value TH1 and the numerical value of the display area 805 in conjunction with the change in the position of the slide bar 806. Therefore, the user can change the threshold value TH1 by sliding the slide bar 806 using a pointing device such as the mouse 357.

[0105] The information processing unit 301 can accept input of a numerical value in the display area 805 using, for example, the keyboard 358. Thus, the user can change the threshold value TH1 by directly inputting a numerical value in the display area 805.

[0106] The information processing unit 301 displays the simulation results when the changed threshold value TH1 is used in the display area 807. That is, the information processing unit 301 accepts a change to the range R1, i.e., the threshold value TH1, in the UI image UI4, estimates the machine resource usage status according to the change using the moving images M1 and M2, and displays the estimation result in the display area 807 in the UI image UI4. The estimation is also a simulation. Moreover, the estimation result is also a simulation result.

[0107] A simulation (estimation) of the machine resource usage will be specifically described. The information processing unit 301 calculates an estimated time for the blind spot level score to exceed the changed threshold TH1, and displays the estimated time relative to the recording time of the video M1 as the operation rate of the camera 202 in the display area 807. The information processing unit 301 also calculates an average value of the usage rate of the CPU 351, an average value of the bandwidth usage of the network NW, and a total power usage relative to the recording time of the video M1, and displays them in the display area 807. At this time, it is preferable that the information processing unit 301 displays the simulation result using the changed threshold TH1 in the display area 807 together with the simulation result using the threshold TH1 before the change.

[0108] The information processing unit 301 displays the moving image M1 in the display area 808, and also displays an image of a skeleton model based on the extracted skeleton coordinate information in an animated form on the moving image M1.

[0109] The information processing unit 301 also displays the analysis results of the motion of the selected part in the display area 809. In the example shown in FIG. 12, the analysis results of the waist bending angle corresponding to the "right waist" and the analysis results of the waist twisting angle are displayed in the display area 809. The analysis process is the same as the flow of the analysis process shown in FIG. 5. In this case, it is assumed that there are no frames in the moving image M2 that have a blind spot level score exceeding the threshold TH1. The information processing unit 301 displays the analysis results of the motion of the selected part as a graph in the display area 809. In this embodiment, the information processing unit 301 displays the analysis results corresponding to the threshold TH1 before the change and the analysis results corresponding to the threshold TH1 after the change, superimposed on the graph. This allows the user to visually compare the analysis results before the change of the threshold TH1 and the analysis results after the change of the threshold TH1.

[0110] When the threshold value TH1 is changed by the user, the information processing unit 301 performs a simulation in conjunction with the change, and changes the display in the display area 807 and the display in the display area 809 according to the results of the simulation.

[0111] In this way, the user can confirm the change in machine resource usage resulting from the change in threshold TH1 by referring to display area 807, and the change in analysis results resulting from the change in threshold TH1 by referring to display area 809. For example, the user can confirm the change in the load on the storage as a change in machine resource usage, and the change in the accuracy of the analysis results as a change in the analysis results.

[0112] There is a trade-off between the load on the machine resources displayed in the display area 807 and the accuracy of the analysis results displayed in the display area 809 with respect to the change in the threshold value TH1. Therefore, the user can search for a threshold value TH1 that suits the purpose of use by changing the threshold value TH1.

[0113] The information processing unit 301 displays a seek bar 811 indicating the time axis when playing back a video in the display area 808. The seek bar 811 corresponds to the time axis of the graph displayed in the display area 809. A slider 810 that moves in conjunction with the playback time of the video is displayed across the display areas 808 and 809.

[0114] The slider 810 can also be operated by the user, and the user can change the playback location by moving the slider 810. By using the slider 810, the user can view the video at the time that the user wants to check in the analysis results. Therefore, the movement of the person P1 that is the basis of the analysis results can be easily confirmed. For example, the cause of the blind spot can be easily confirmed.

[0115] As described above, the user can change the threshold value TH1 for each part of the person P1 to be analyzed, taking into consideration the usage situation, conditions, etc. Furthermore, the user can visually change the threshold value TH1 via the UI image UI4.

[0116] The data of the threshold value TH1 changed by the above process is overwritten and saved in the SSD 354. In this manner, the user can set the threshold value TH1 for each part according to the situation, environment, or purpose, using a GUI such as the UI image UI4 shown in FIG.

[0117] In this manner, according to this embodiment, camera 201 is the main camera, camera 202 is the sub-camera, and recording of video image M2 obtained from camera 202 can be switched on / off in real time. This makes it possible to reduce the utilization rate of machine resources, such as CPU 351, storage such as SSD 354, and the bandwidth of network NW.

[0118] The present invention is not limited to the above-described embodiments, and many modifications are possible within the technical concept of the present invention. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0119] The above-mentioned information processing device 300 can be configured by a computer such as a desktop PC, a laptop PC, a tablet PC, or a smartphone. In addition, the display device constituting the display unit is the display 356, and the input device constituting the input unit is a mouse and a keyboard, but the present invention is not limited to these. For example, the information processing device 300 may include a touch panel display that combines a display unit and an input unit.

[0120] In the above embodiment, the CPU 351 functions as the information processing unit 301, but the present invention is not limited to this. The functions of the information processing unit 301 may be realized by a plurality of computers, i.e., a plurality of CPUs. For example, in the case where the camera 201 has a built-in CPU, such as an industrial camera, the CPU 351 and the CPU built in the camera 201 may cooperate to function as the information processing unit 301. For example, the CPU built in the camera 201 may perform the information processing of step S103 in FIG. 4. By having a plurality of CPUs take charge of the functions of the information processing unit 301, the information processing unit 301 can perform processing more quickly.

[0121] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0122] The disclosure of the above embodiments includes the following sections.

[0123] (Section 1) A first imaging unit; A second imaging unit disposed at a position different from the first imaging unit; An information processing unit, The information processing unit includes: obtaining a score indicating an imaging state of a predetermined part of a person appearing in a first captured image obtained by imaging the first imaging unit; a first operation of recording a second captured image obtained by imaging the second imaging unit or a second operation of not recording the second captured image based on the score; An information processing system comprising:

[0124] (Section 2) The information processing unit includes: analyzing a movement of the person based on at least one of the first captured image and the second captured image; 2. The information processing system according to item 1,

[0125] (Section 3) The information processing unit includes: If the score is within a predetermined range, execute the second action, and if the score is outside the predetermined range, execute the first action. 3. The information processing system according to item 1 or 2,

[0126] (Section 4) The information processing unit includes: a user interface image for accepting changes to the predetermined range is displayed on a display unit; 4. The information processing system according to item 3,

[0127] (Section 5) The information processing unit includes: estimating a usage status of machine resources in response to the change in the predetermined range, and displaying the estimation result on the user interface image; 5. The information processing system according to item 4,

[0128] (Section 6) The imaging state of the predetermined part is a degree of hiding of the predetermined part. 6. The information processing system according to any one of items 1 to 5,

[0129] (Section 7) the information processing unit outputs information regarding the posture of the person as an analysis result. 3. The information processing system according to item 2,

[0130] (Section 8) The information processing unit includes: A first moving image including a plurality of the first captured images captured by the first imaging unit can be acquired, A second moving image including a plurality of the second captured images captured by the second imaging unit can be acquired. 8. The information processing system according to any one of items 1 to 7,

[0131] (Section 9) The information processing unit includes: acquiring the score each time the first captured image in the first moving image is acquired. 9. The information processing system according to item 8,

[0132] (Section 10) The information processing unit includes: Display the machine resource usage status on the display unit. 10. The information processing system according to any one of items 1 to 9,

[0133] (Section 11) The information processing unit includes: The machine resource usage status is displayed on the display. the machine resource usage status includes information related to a recording time of the second video; 10. The information processing system according to item 8 or 9,

[0134] (Section 12) the first imaging unit, the second imaging unit, and the information processing unit are connected via a network, The information processing unit includes: The machine resource usage status is displayed on the display. The machine resource usage status includes the network usage status. 10. The information processing system according to item 8 or 9,

[0135] (Section 13) The information processing unit includes: causing the second imaging unit to perform imaging in synchronization with the first imaging unit; 13. The information processing system according to any one of items 1 to 12,

[0136] (Section 14) The information processing unit includes: If the second captured image synchronized with the first captured image has not been recorded, the movement of the person is analyzed based on the first captured image. Item 14. The information processing system according to item 13,

[0137] (Section 15) The information processing unit includes: If the second captured image synchronized with the first captured image has been recorded, the movement of the person is analyzed based on the first captured image and / or the second captured image. 15. The information processing system according to item 13 or 14,

[0138] (Section 16) The information processing unit acquires part information regarding a position of the predetermined part based on the first captured image or the second captured image. 16. The information processing system according to any one of items 1 to 15,

[0139] (Section 17) The site information is skeletal coordinate information. 17. The information processing system according to item 16,

[0140] (Section 18) The information processing unit displays an image corresponding to the part information on a display unit. 18. The information processing system according to item 16 or 17,

[0141] (Section 19) An information processing device including an information processing unit, The information processing unit includes: obtaining a score indicating an imaging state of a predetermined part of a person appearing in a first captured image obtained by imaging with the first imaging unit; and executing a first operation of recording a second captured image obtained by imaging with the second imaging unit or a second operation of not recording the second captured image based on the score. 23. An information processing apparatus comprising:

[0142] (Section 20) 1. An information processing method, comprising: obtaining a score indicating an imaging state of a predetermined part of a person appearing in a first captured image obtained by imaging with the first imaging unit; and executing a first operation of recording a second captured image obtained by imaging with the second imaging unit or a second operation of not recording the second captured image based on the score. 23. An information processing method comprising:

[0143] (Section 21) Item 19. A method for manufacturing an article, comprising the steps of: manufacturing an article based on a result of information processing by the information processing system according to any one of items 1 to 18.

[0144] (Section 22) 21. A program for causing a computer to execute the information processing method according to item 20.

[0145] (Section 23) Item 23. A computer-readable recording medium having the program according to item 22 recorded thereon. [Explanation of symbols]

[0146] 100...information processing system, 201...camera (first imaging section), 202...camera (second imaging section), 300...information processing device, 301...information processing section

Claims

1. A first imaging unit, A second imaging unit disposed at a position different from that of the first imaging unit, An information processing unit, and is provided with, The information processing unit, Obtains a score indicating the degree of concealment of a predetermined part of a person shown in a first captured image obtained by capturing an image with the first imaging unit, Based on the score, executes a first operation of recording a second captured image obtained by capturing an image with the second imaging unit, or a second operation of not recording the second captured image, An information processing system characterized by the above.

2. The information processing unit, Analyzes the movement of the person based on at least one of the first captured image and the second captured image, The information processing system according to claim 1, characterized by the above.

3. The information processing unit, If the score is within a predetermined range, the second operation is executed, and if the score is outside the predetermined range, the first operation is executed, The information processing system according to claim 1, characterized by the above.

4. The information processing unit, Displays a user interface image for accepting a change of the predetermined range on a display unit, The information processing system according to claim 3, characterized by the above.

5. The information processing unit, Estimates the usage status of machine resources according to the change of the predetermined range, and displays the estimation result on the user interface image, The information processing system according to claim 4, characterized by the above.

6. The information processing unit outputs information regarding the posture of the person as an analysis result, The information processing system according to claim 2, characterized by the above.

7. The information processing unit, Can acquire a first moving image including a plurality of the first captured images captured by the first imaging unit, Can acquire a second moving image including a plurality of the second captured images captured by the second imaging unit, The information processing system according to claim 1, characterized by the above.

8. The information processing unit, Acquires the score each time the first captured image in the first moving image is acquired, The information processing system according to claim 7, characterized by the above.

9. The information processing unit, Displays the usage status of machine resources on a display unit, The information processing system according to claim 1, characterized by the above.

10. The information processing unit, Displays the usage status of machine resources on a display unit, The usage status of the machine resources includes information related to the recording time of the second moving image, The information processing system according to claim 7, characterized by the above.

11. The first imaging unit and the second imaging unit are connected to the information processing unit via a network. The information processing unit displays the usage status of machine resources on a display unit, wherein the usage status of the machine resources includes the usage status of the network. The information processing system according to claim 7, characterized in that.

12. The information processing unit causes the second imaging unit to perform imaging in synchronization with the first imaging unit. The information processing system according to claim 1, characterized in that.

13. The information processing unit analyzes the actions of the person based on the first captured image if the second captured image synchronized with the first captured image is not recorded. The information processing system according to claim 12, characterized in that.

14. The information processing unit analyzes the actions of the person based on the first captured image and / or the second captured image if the second captured image synchronized with the first captured image is recorded. The information processing system according to claim 12, characterized in that.

15. The information processing unit obtains part information regarding the position of the predetermined part based on the first captured image or the second captured image. The information processing system according to claim 1, characterized in that.

16. The part information is skeletal coordinate information. The information processing system according to claim 15, characterized in that.

17. The information processing unit displays an image corresponding to the part information on a display unit. The information processing system according to claim 15, characterized in that.

18. An information processing apparatus including an information processing unit, wherein the information processing unit obtains a score indicating the degree of concealment of a predetermined part of a person shown in a first captured image obtained by imaging of a first imaging unit, and based on the score, executes a first operation of recording a second captured image obtained by imaging of a second imaging unit or a second operation of not recording the second captured image. The information processing apparatus, characterized in that.

19. An information processing method, obtaining a score indicating the degree of concealment of a predetermined part of a person shown in a first captured image obtained by imaging of a first imaging unit, and based on the score, executing a first operation of recording a second captured image obtained by imaging of a second imaging unit or a second operation of not recording the second captured image. The information processing method, characterized in that.

20. A method for manufacturing an article, characterized by manufacturing the article based on the result of information processing by the information processing system according to any one of claims 1 to 17.

21. A program for causing a computer to execute the information processing method according to claim 19.

22. A computer-readable recording medium having recorded thereon the program according to claim 21.

Images