Information processing device, information processing method, and program

Abstract

To provide an information processing device, method and program for reducing a load to a user relating to an analysis of a work execution situation.SOLUTION: In an information processing system in which a server device, a plurality of terminal devices, and an imaging device are connected through a network so as to be able to transmit and receive information to / from one another, the server device 100 being a kind of an information processing device includes an input / output control part 102 for receiving designation of a plurality of subjects relating to execution of work of a management object, and designation of work to be allocated to the plurality of subjects, and a management part 106 for acquiring information related to an execution period of the designated work, and recording records related to management of the work of each of the plurality of subjects by associating the designated work with the execution period of the work with the information acquired as common information for each of the plurality of designated subjects.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In work sites of various operations including manufacturing, in order to improve productivity, improve product quality, improve service quality, etc., it is required to identify problems and solutions to the problems. Against this background, various technologies for assisting managers in analyzing the implementation status of work have been studied. For example, Patent Document 1 discloses an example of a technology for assisting managers in analyzing the implementation status of work by using data of moving images according to the imaging results of a work site.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, even in a situation where a tool for assisting in the analysis of work status is used, for example, the preparation before analysis such as registering information to be analyzed is still time-consuming, and the load related to the analysis of the implementation status of work still tends to be high. In particular, as the number of analysis targets increases, the labor required to register information about the analysis targets proportionally increases, and the labor required to ensure the consistency of the information registered among multiple analysis targets further tends to increase. Against this background, there is a demand for the introduction of a mechanism that further reduces the burden on users (for example, managers) related to the analysis of the implementation status of work.

[0005] In view of the above problems, an object of the present invention is to further reduce the burden on users related to the analysis of the implementation status of work. [Means for solving the problem]

[0006] The information processing device according to the present invention includes a receiving means for receiving the designation of multiple entities from a series of entities to be synchronized for the execution of managed tasks, and the designation of tasks to be commonly assigned to the multiple entities, and the start date and time of the tasks as the execution period of the designated tasks. The first timestamp indicating and The work End date and time The second timestamp indicates A means for obtaining the acquisition, and for each of the specified multiple entities, Includes subject identification information, task identification information, first timestamp, and second timestamp as items. record The work identification information, the first timestamp, and the second timestamp are generated in synchronization, and the records generated for each of the multiple entities are stored together in the storage unit. A recording means for recording, The recording means does not record the record if at least a portion of the implementation period specified by the first and second timestamps associated with the record to be registered overlaps with at least a portion of the implementation period associated with other records already recorded, and if the entity associated with the record overlaps with the entity associated with the other records. The first timestamp or the second timestamp is obtained by detecting the action or state of at least one of the multiple entities from a video image of the work environment in which the specified work is being performed, and based on the detection result, identifying the timing indicating the start or end of the action or state. . [Effects of the Invention]

[0007] According to the present invention, it is possible to further reduce the burden on users in analyzing the status of work execution. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram shows an example of the system configuration of an information processing system. [Figure 2] This diagram shows an example of the hardware configuration of an information processing device. [Figure 3] This is a functional block diagram showing an example of the functional configuration of an information processing system. [Figure 4] This diagram shows an example of a screen used for managing the progress of work. [Figure 5] This diagram illustrates the functions related to managing the status of work completion. [Figure 6] This diagram shows an example of the management interface within the administration screen. [Figure 7] This diagram shows an example of a screen for registering the status of work performed. [Figure 8] This is a diagram illustrating the functions related to the playback of moving images. [Figure 9] This diagram illustrates the function related to the output of registered information. [Figure 10] FIG. 1 is a diagram showing an example of a screen for using a function related to image analysis. [Figure 11] FIG. 2 is a diagram showing an example of a screen for adding or updating settings related to image analysis. [Figure 12] FIG. 3 is a diagram showing an example of a screen for specifying setting information for analysis processing. [Figure 13] FIG. 4 is a diagram showing an example of the application status of analysis processing to an image. [Figure 14] FIG. 5 is a diagram for explaining Example 1. [Figure 15] FIG. 6 is a diagram for explaining Example 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0010] <System Configuration> Referring to FIG. 1, an example of the system configuration of an information processing system according to an embodiment of the present disclosure will be described. The information processing system 1 according to the present embodiment includes a server device 100 and one or more terminal devices 200. The terminal devices 200a and 200b shown in FIG. 1 each show an example of the terminal device 200. In the following description, when the terminal devices 200a and 200b are not particularly distinguished from each other, they are simply referred to as the terminal device 200. The information processing system 1 may also include one or more imaging devices 300. The imaging devices 300a and 300b shown in FIG. 1 each show an example of the imaging device 300. In the following description, when the imaging devices 300a and 300b are not particularly distinguished from each other, they are simply referred to as the imaging device 300.

[0011] The server device 100, each terminal device 200, and the imaging device 300 are connected to each other via a network N1 so as to be able to transmit and receive information. Note that the type of the network N1 is not particularly limited. As a specific example, the network N1 may be constituted by the Internet, a dedicated line, a LAN (Local Area Network), a WAN (Wide Area Network), or the like. Further, the network N1 may be constituted by a wired network, or may be constituted by a wireless network such as a network based on communication standards such as 5G, LTE (Long Term Evolution), and Wi-Fi (registered trademark). Further, the network N1 may include a plurality of networks, and as a part of the networks, a network of a different type from other networks may be applied. Further, it is only necessary that the communication between the various information processing devices described above is logically established, and physically, the communication between the various information processing devices may be relayed by other communication devices or the like.

[0012] The terminal device 200 serves as an interface for receiving inputs (e.g., various instructions) from the user and presenting various information (e.g., feedback, etc.) to the user. As a specific example, the terminal device 200 may receive data from the server device 100 described later via the network and present information based on the data to the user via a predetermined output device (e.g., a display, etc.). Further, the terminal device 200 may recognize an instruction from the user based on an operation received from the user via a predetermined input device (e.g., a touch panel, etc.) and transmit information corresponding to the instruction to the server device 100 via the network. Thereby, the server device 100 can recognize an instruction from the user and execute processing corresponding to the instruction. Further, details of an example of the above feedback will be described separately later in conjunction with the description of the examples shown in FIGS. 5 to 7. The terminal device 200 can be realized by an information processing device having a communication function such as a so-called smartphone, a tablet terminal, and a PC (Personal Computer).

[0013] The imaging device 300 captures images within its imaging range (in other words, within its field of view) and transmits still image or moving image data corresponding to the imaging results to a predetermined destination (for example, a server device 100) via the network N1. For convenience, in the following explanation, when there is no particular distinction between still images and moving images, they will simply be referred to as images, and the data of such images will also be referred to as image data. As a specific example, the device may capture images of the work environment in which an operator performs various tasks and transmit the image data corresponding to the imaging results to the server device 100.

[0014] The server device 100 provides various functions to support the management and analysis of the work performed by the workers being managed by the administrator (hereinafter simply referred to as the administrator). As a specific example, the server device 100 provides a UI (User Interface) that enables the management of the work performed by each of the multiple workers being managed, and the management of the usage status of various equipment used by each worker when performing their work, with simpler operation. As another example, the server device 100 may also provide functions to support the analysis when the administrator analyzes images (e.g., moving images) corresponding to the imaging results of the work environment in which the workers perform various tasks by the imaging device 300. Furthermore, the server device 100 may analyze the images and use the results of the analysis to support the management and analysis when the administrator manages and analyzes the work performed by the workers. Details of the various functions of the server device 100 described above will be described separately.

[0015] The configuration shown in Figure 1 is merely an example and does not necessarily limit the system configuration of the information processing system 1 according to this embodiment. As a specific example, the server device 100 may be implemented as a standalone device. In this case, the server device 100 may accept input of various information from the user and may present various information to the user. Alternatively, the components corresponding to the server device 100 may be implemented by multiple devices working together. As a specific example, the components corresponding to the server device 100 may be implemented as a so-called cloud service. In this case, the cloud service may be implemented by multiple server devices working together. Furthermore, if the server device 100 can acquire image data corresponding to the imaging results from the imaging device 300, the method of acquisition is not necessarily limited to acquiring the data via a network. As a specific example, the server device 100 and the imaging device 300 may be connected via a transmission path such as a cable, and the image data corresponding to the imaging results from the imaging device 300 may be transferred to the server device 100 via the said transmission path. Alternatively, the image data corresponding to the imaging results from the imaging device 300 may be read into the server device 100 by using a recording medium that can be attached to or detached from the server device 100 or the imaging device 300.

[0016] An example of the system configuration of an information processing system according to one embodiment of the present disclosure has been described above with reference to Figure 1.

[0017] <Hardware Configuration> Referring to Figure 2, an example of the hardware configuration of an information processing device 900 applicable as various information processing devices (for example, a server device 100 or a terminal device 200, etc.) constituting the information processing system 1 according to this embodiment shown in Figure 1 will be described. The information processing device 900 includes a CPU (Central Processing Unit) 910, a ROM (Read Only Memory) 920, a RAM (Random Access Memory) 930, an auxiliary storage device 940, and a network I / F 970. The information processing device 900 may also include at least one of an output device 950 and an input device 960. The CPU 910, ROM 920, RAM 930, auxiliary storage device 940, output device 950, input device 960, and network I / F 970 are interconnected via a bus 980.

[0018] The CPU 910 is a central processing unit that controls various operations of the information processing unit 900. For example, the CPU 910 may control the operation of the entire information processing unit 900. The ROM 920 stores control programs, boot programs, and other programs that can be executed by the CPU 910. The RAM 930 is the main memory of the CPU 910 and is used as a work area or a temporary storage area for deploying various programs.

[0019] The auxiliary storage device 940 stores various data and programs. The auxiliary storage device 940 is implemented by a storage device capable of temporarily or permanently storing various data, such as an HDD (Hard Disk Drive) or non-volatile memory such as an SSD (Solid State Drive).

[0020] The output device 950 is a device that outputs various types of information and is used to present various types of information to the user. In this embodiment, the output device 950 is implemented by a display device such as a display. The output device 950 presents information to the user by displaying various types of display information. However, as another example, the output device 950 may be implemented by an acoustic output device that outputs sounds such as voice or electronic sounds. In this case, the output device 950 presents information to the user by outputting sounds such as voice or telegraph. Furthermore, the device applied as the output device 950 may be appropriately changed depending on the medium used to present information to the user. Note that the output device 950 corresponds to an example of an "output unit" used to present various types of information.

[0021] The input device 960 is used to receive various instructions from the user. In this embodiment, the input device 960 includes input devices such as a mouse, keyboard, and touch panel. However, as another example, the input device 960 may include a sound collection device such as a microphone to collect the voice spoken by the user. In this case, various analysis processes such as acoustic analysis and natural language processing are applied to the collected voice so that the content of the voice is recognized as an instruction from the user. Furthermore, the device applied as the input device 960 may be changed as appropriate depending on the method of recognizing the user's instructions. In addition, multiple types of devices may be applied as the input device 960.

[0022] The network interface 970 is used for communication with external devices via a network. The device used as the network interface 970 may be changed as appropriate depending on the type of communication path and the applicable communication method.

[0023] The program for the information processing device 900 may be provided to the information processing device 900, for example, by a recording medium such as a CD-ROM, or it may be downloaded via a network or the like. When the program for the information processing device 900 is provided by a recording medium, the program recorded on the recording medium is installed on the auxiliary storage device 940 when the recording medium is set in a predetermined drive device.

[0024] Furthermore, the configuration shown in Figure 2 is merely an example and does not necessarily limit the hardware configuration of the information processing device constituting the information processing system 1 according to this embodiment. For example, some components such as the input device 960 and the output device 950 may not be included. Another example is that configurations appropriate to the functions realized by the information processing device 900 may be added as needed.

[0025] The above describes an example of the hardware configuration of an information processing device 900 that can be applied as various information processing devices constituting the information processing system 1 according to the present embodiment shown in Figure 1, with reference to Figure 2.

[0026] <Functional Configuration> Referring to Figure 3, an example of the functional configuration of the information processing system 1 according to this embodiment will be described, with particular attention paid to the configuration of the server device 100. The server device 100 includes a communication unit 101, an input / output control unit 102, an input analysis unit 103, a playback control unit 104, an image analysis unit 105, a management unit 106, and a storage unit 107.

[0027] The communication unit 101 is a communication interface for each component of the server device 100 to send and receive information with other devices (e.g., terminal devices 200) via the network N1. The communication unit 101 can be implemented, for example, by a network I / F 970. In the following description, unless otherwise specified, when each component of the server device 100 sends and receives information with other devices, it will be assumed that such information is sent and received via the communication unit 101.

[0028] The memory unit 107 schematically represents a memory area for storing various data and programs. For example, the memory unit 107 may store data and programs for each component of the server device 100 to execute processing. Furthermore, the storage unit 107 may store image data that the playback control unit 104 (described later) will process, image data that the image analysis unit 105 (described later) will analyze, and various types of information that the management unit 106 (described later) will manage.

[0029] The input / output control unit 102 performs various processes related to presenting various information to the user and receiving information input from the user (e.g., instructions). As a specific example, the input / output control unit 102 performs various processes related to presenting a UI (User Interface) as described later, with reference to Figures 4 to 13, and receiving input via the UI. An example of the UI will be described in detail separately, along with an example of the processing performed in conjunction with input / output via the UI. The part of the input / output control unit 102 that specifically handles receiving information input from the user corresponds to an example of a "reception means".

[0030] The input analysis unit 103 recognizes user instructions based on the information received by the input / output control unit 102 and outputs information corresponding to the result of the recognition to a predetermined output destination (for example, the playback control unit 104, image analysis unit 105, and management unit 106, which will be described later). For example, when the input analysis unit 103 receives input from the user via a UI presented to the user, it may recognize instructions from the user based on the received input. As a specific example, when an input interface such as a button presented on the UI receives an operation from the user, the input analysis unit 103 may recognize the instruction associated with the input interface and output the instruction to a predetermined output destination. As another example, the input analysis unit 103 may output information (for example, parameters, etc.) entered by the user in an input area presented on the UI to a predetermined output destination that uses that information for processing.

[0031] The playback control unit 104 performs processing related to the playback of an image corresponding to a desired image data. For example, the playback control unit 104 may extract image data stored in the storage unit 107 and perform processing related to the playback of an image (e.g., a moving image) based on that image data. Alternatively, as another example, the playback control unit 104 may receive image data corresponding to the imaging result from a predetermined imaging device (e.g., imaging device 300) and perform processing related to the playback of an image based on that image data. Furthermore, the playback control unit 104 may control the processing related to the playback of multiple image data. As a specific example, the playback control unit 104 may concatenate and play back multiple images corresponding to imaging results from the same imaging device at different times, based on information indicating the imaging time (e.g., a timestamp) associated with the image data corresponding to these images. As another example, the playback control unit 104 may synchronize and play back multiple images corresponding to imaging results from multiple different imaging devices, based on information indicating the imaging time associated with the image data corresponding to these images. Furthermore, the playback control unit 104 may control the image playback process based on information corresponding to user instructions output from the input analysis unit 103. As a specific example, when playing back a video, the playback control unit 104 may start playback from a time instructed by the user. As described above, the playback control unit 104 may perform processing related to the playback of an image corresponding to the desired image data and output the result of the processing to the input / output control unit 102. This allows the input / output control unit 102 to present the played-back image to the user by displaying it in a predetermined output destination (for example, a predetermined display area in the UI).

[0032] The image analysis unit 105 performs various analyses on the image represented by the desired image data (for example, the image reconstructed based on the image data) and outputs the results of the analysis to a predetermined output destination (for example, the management unit 106, which will be described later). For example, the image analysis unit 105 may extract image data stored in the storage unit 107 and perform a predetermined analysis on the image represented by that image data. Alternatively, as another example, the image analysis unit 105 may receive image data corresponding to the imaging results from a predetermined imaging device (for example, the imaging device 300) and perform a predetermined analysis on the image represented by that image data.

[0033] The analysis performed by the image analysis unit 105 on an image may be modified as appropriate depending on the purpose of the analysis. As a specific example, the image analysis unit 105 may detect changes in the environment being captured by detecting changes in the image shown by the target image data. As another example, the image analysis unit 105 may recognize a subject captured in an image by performing image analysis on the image shown by the target image data. As yet another example, the image analysis unit 105 may recognize the actions of a subject captured in a video by performing image analysis on the video shown by the target image data. For example, based on the results of the above image analysis, the image analysis unit 105 may detect the timing when the subject (e.g., a person or equipment) that is the main subject of the action to be analyzed in the video started the action, and the timing when the subject finished the action.

[0034] Furthermore, the methods that the image analysis unit 105 applies to image analysis are not particularly limited. For example, the image analysis unit 105 may detect changes in the environment being captured based on changes in pixels in the image (e.g., changes in brightness values) indicated by the target image data. Another example is that the image analysis unit 105 may extract feature quantities related to the shape and color characteristics of the subject from the image data indicated by the target image data, and use these feature quantities to recognize the subject captured in the image. Yet another example is that the image analysis unit 105 may use a recognizer or classifier built based on so-called machine learning for image analysis. As a specific example, a recognizer may be constructed using a pair of an image and information about the subject captured in that image as training data. By using such a recognizer, it becomes possible to recognize the subject captured in the input image.

[0035] The management unit 106 executes various processes related to managing the status of work performed by the workers being managed. The management unit 106 may also execute processes related to managing the usage status of various equipment used by workers during work (in other words, managing the status of work performed using various equipment). For example, the management unit 106 manages information relating to the workers being managed, the work being performed by those workers, and the duration of that work, in response to instructions from the administrator via the UI presented by the input / output control unit 102. The management unit 106 may also manage information relating to the equipment used during work (e.g., cranes, lifters, etc.), the work itself, and the duration of that work. The management unit 106 stores this information related to managing the status of work performed by workers and information related to managing the usage status of various equipment used by workers during work in a predetermined storage area (e.g., storage unit 107, etc.). In this way, the management unit 106 executes various processes related to managing the status of work performed by the work entities, such as workers and the equipment used by those workers. Details regarding the processes related to managing the status of work performed by the workers being managed, and the processes related to managing the usage status of various equipment used by the workers during work, will be described separately in detail along with an explanation of the UI used for such management. Furthermore, within the management unit 106, the part related to acquiring information regarding the period of the above-mentioned work corresponds to an example of an "acquisition means," and the part that stores the information related to the above-mentioned various management in a predetermined storage area corresponds to an example of a "recording means."

[0036] It should be noted that the above configuration is merely an example, and the functional configuration of the server device 100 is not necessarily limited to the example shown in Figure 3. For example, a series of components of the server device 100 may be realized by the cooperation of multiple devices. As a specific example, some of the components of the server device 100 may be externally connected to the server device 100. As another example, the processing load of at least some of the components of the server device 100 may be distributed among multiple devices.

[0037] As described above, with reference to FIG. 3, an example of the functional configuration of the information processing system 1 according to the present embodiment has been described, particularly focusing on the configuration of the server device 100.

[0038] <UI and Processing> With reference to FIGS. 4 to 14, an example of the UI provided by the information processing system according to the present embodiment and the processing executed based on instructions from a user (administrator) via the UI will be described below. In the following description, for convenience, it is assumed that the server device 100 presents the UI to the administrator, accepts instructions from the administrator via the UI, and executes processing according to the instructions, and various explanations will be made accordingly.

[0039] First, the UI shown in FIG. 4 will be described. The management screen V100 shown in FIG. 4 is an example of a management screen that serves as a UI for presenting information to the administrator and accepting instructions from the administrator when the information processing system according to the present embodiment provides various functions to the administrator. For example, the management screen V100 serves as an interface when the administrator uses a function for assisting in the analysis of an image (e.g., a moving image) according to the imaging result of the working environment where an operator performs various operations. As another example, the management screen V100 serves as an interface when the administrator uses a function for assisting in the management of the implementation status of various operations by the main bodies of the operations, such as operators or equipment used by the operators.

[0040] Here, the screen configuration of the management screen V100 will be described in more detail. The management screen V100 includes display areas V101, V103, and V104, a management interface V102, and an operation interface V105.

[0041] The display area V104 is a display area where an image (e.g., a moving image according to the imaging result of the working environment) used for analyzing the implementation status of operations by the main bodies such as operators or equipment is displayed. Furthermore, display area V103 is a display area that displays information about the image data that serves as the source for the image displayed in display area V104. As a specific example, when the playback result of a video is displayed in display area V104, management information related to the target video, such as the total number of frames, frame number, and file name, as well as information about the data of the video, will be displayed in display area V103. Furthermore, display areas V103 and V104 may be provided individually for each image to be displayed (in other words, for each image data subject to playback processing). For example, in the example shown in Figure 4, two moving images are to be displayed, and for each of these two moving images, display area V103 (i.e., display areas V103a and V103b) and display area V104 (i.e., display areas V104a and V104b) are provided. Furthermore, if multiple images are to be displayed, synchronization may be performed when displaying those multiple images. This makes it possible, for example, to display images captured by multiple different imaging devices at a desired date and time individually in display areas V104a and V104b, respectively.

[0042] Display area V101 displays various management information related to the display of images in display area V104. For example, display area V101 may display information regarding the date and time when the image (e.g., video) displayed in display area V104 was captured. As a specific example, if multiple synchronized video images are displayed in display areas V104a and V104b, the date and time corresponding to the frames displayed in display areas V104a and V104b may be displayed in display area V101.

[0043] The operation interface V105 is an input interface for receiving various instructions from the administrator regarding the display of images in the display area V104 (for example, display areas V104a and V104b). As a specific example, if the object to be displayed in the display area V104 is a moving image, the operation interface V105 may include an input interface for receiving instructions from the administrator regarding playback and stopping of the moving image, jumping to a frame corresponding to a predetermined timing, and adjusting the playback speed. Furthermore, the operation interface V105 may also include an input interface for receiving instructions from the administrator regarding assigning an index to a desired frame in the moving image and jumping to a frame to which such index has been assigned.

[0044] The management interface V102 is an input / output interface for displaying information related to the management of the status of various tasks performed by registered work entities such as workers and equipment, and for receiving instructions from managers regarding such management. For example, the management interface V102 displays information about the tasks assigned to each managed entity (i.e., workers and equipment), and information about the duration of those tasks (e.g., start date and time, end date and time, length of duration, etc.). The management interface V102 may also display information about the tasks assigned to the target entities. For example, in the example shown in Figure 4, information indicating the content of the tasks is displayed as information about the tasks assigned to each entity.

[0045] Furthermore, the management screen V100 shown in Figure 4 may also have a display area V106 for displaying supplementary information. For example, in the example shown in Figure 4, the display area V106 displays an image showing the layout of the work environment in which the worker performs various tasks. The information displayed in the display area V106 may also be switched as appropriate according to user instructions or information available at the time.

[0046] Here, we will individually explain the functions available through the V100 administration screen.

[0047] (Information registration and management) First, referring to Figures 5 to 7, we will explain an example of registering information related to the management of the implementation status of various tasks by the main entities of the work, such as workers and equipment, and providing feedback according to the registration status, along with the operation method and the processes executed in response to said operation.

[0048] Figure 5 is a diagram illustrating the overview of the functions of the server device 100 related to managing the status of work performed by each entity, schematically showing the status of work performed by multiple entities that are subject to management, such as workers and equipment. In Figure 5, the horizontal axis represents time, with the left direction representing the past and the right direction representing the future. In the information processing system according to this embodiment, the administrator registers information on the work performed by the main entities of the work, such as workers and equipment, and the period during which the work is being performed, while viewing moving images corresponding to the imaging results of the work environment. The server device 100 provides a UI that enables the administrator to register information used for managing the work status of each entity in a consistent and efficient manner.

[0049] Specifically, administrators register data related to the management of each entity's work (hereinafter, each piece of data will also be referred to as a record) by specifying the entity to be registered, the task to be assigned to that entity, and the period for which that task will be performed, using the provided UI. As a concrete example, in the example shown in Figure 5, "Worker B," who is the main performer of the work, is performing "slinging work" as one of his tasks, and after performing the slinging work, he is performing "crane work" as another task. In other words, for "Worker B," records related to the management of "slinging work" and records related to the management of "crane work" will be recorded according to the user's instructions for information registration.

[0050] Furthermore, depending on the nature of the work, situations may arise where multiple entities perform the same task. For example, in situations where a worker uses equipment such as a crane to perform a task, the worker and the equipment will be assigned the same task. As a specific example, in the example shown in Figure 5, "Worker A" is using a "crane" when performing "crane work." Therefore, records related to the management of "crane work" for the same period will be recorded in common for both "Worker A," who is the main performer of the work, and the "crane." Similarly, "Worker C" uses the "lifter" when performing the "lifter work." Therefore, records related to the management of the "lifter work" for the same period will be recorded in common for both "Worker C," who is the main performer of the work, and the "lifter." Furthermore, "Worker B" uses the "crane" in the "crane operation" that is performed after the "rigging operation." Therefore, records related to the management of the "crane operation" for the same period will be recorded in common for both "Worker B," who is the main performer of the work, and the "crane." In light of these circumstances, the server device 100 supports the administrator's registration operations so that, in situations where multiple entities perform a common task, records related to the management of the status of the task performed by each of those entities can be registered more efficiently.

[0051] Here, with reference to Figures 6 and 7, we will explain the operations related to registering information and the processing performed by the server device 100 upon receiving said operations, in relation to the "lifter operation" performed by "worker C" using the "lifter".

[0052] For example, Figure 6(a) is a diagram showing an example of the state of the management interface V102 in the management screen V100, as explained with reference to Figure 4, and shows the registration status of information regarding the status of work performed by each entity that has been pre-registered as a managed entity. Specifically, the management interface V102 displays information about a set of entities that have been pre-registered in a list as entities to be managed. Furthermore, entities to be managed may be managed individually according to their type. In this case, for example, a separate list may be generated for each type of entity. As a concrete example, a separate list of workers and a separate list of the equipment used by those workers may be generated. By utilizing such a mechanism, it becomes possible to display to the management interface V102 the registration status of information regarding the work performed by each type of entity.

[0053] The administrator can select a desired entity from a set of entities presented on the management interface V102, thereby accessing a registration screen for registering information about the work performed by that entity. As a specific example, the example shown in Figure 6(a) illustrates the state before information is registered regarding the "lifter work" performed by "worker C" using the "lifter," and no information has been registered for "worker C" or the "lifter." In this state, if the "worker C" item is selected by clicking or other means, a screen related to assigning work to "worker C" (in other words, a screen related to registering information about the work performance status) will be displayed.

[0054] The registration screen V200 shown in Figure 7 is an example of a screen for receiving instructions from an administrator regarding the assignment of work to a target entity (in other words, instructions regarding the registration of the status of work performed by the entity). In the example shown in Figure 7, "Worker C" is the target of information registration, and information indicating Worker C is displayed in the "Worker" field. The registration screen V200 presents an input interface for receiving information from an administrator regarding the work to be assigned to the target entity (for example, Worker C) and information regarding the period during which the work will be performed. As a specific example, in the example shown in Figure 7, it is possible to accept the specification of "work content," which indicates the nature of the work, as information regarding the work to be assigned to the target entity. In addition, in the example shown in Figure 7, it is possible to accept the specification of "start date and time," which indicates the start time of the work, and "end date and time," which indicates the end time of the work, as information regarding the duration of the work.

[0055] Furthermore, the registration screen V200 includes an input area V201 for specifying other entities to be synchronized. If an entity other than the entity targeted for information registration on the registration screen V200 is specified in the input area V201, the information specified via the registration screen V200 will be registered not only for the target entity but also for that other entity. Furthermore, as mentioned above, the entities to be entered into input area V201 may be registered in advance in a list as entities to be managed, and such lists may be generated for each type of entity. In this case, the administrator only needs to select the candidate to be entered into input area V201 from among the candidates for entities registered in the list. For example, in the example shown in Figure 7, "Worker C" is the target of information registration, and information regarding the performance of "lifter work" is entered, with "lifter" specified for input area V201. In this case, the server device 100 registers information regarding the performance of "lifter work" by "Worker C" based on the information entered via the registration screen V200, and also registers information regarding the performance of "lifter work" with the same content for "lifter".

[0056] Figure 6(b) shows another example of the state of the management interface V102 in the management screen V100, as explained with reference to Figure 4, and shows the state after information has been registered regarding the execution of "lifter work" by "worker C" using the "lifter". As can be seen by comparing Figure 6(b) and Figure 6(a), in Figure 6(b), record V1024 regarding the execution of "lifter work" by "worker C" and record V1023 regarding the execution of "lifter work" by the "lifter" are recorded. In addition, records V1024 and V1023 each contain common information regarding the work to be performed (i.e., "lifter work") and information regarding the work period.

[0057] Furthermore, when registering information, the server device 100 may verify the consistency between the information to be registered and other already registered information, and may restrict the registration of information if such consistency cannot be ensured. As a specific example, in the example shown in Figure 5, the registration of "lifter work" using a "lifter" after "crane work" is instructed for "worker A". On the other hand, the implementation period of the "lifter work" performed by "worker A" that is to be registered overlaps with the implementation period of the "lifter work" performed by "worker C" using the same "lifter". During this overlapping period, even though "worker C" is using the "lifter" when performing the "lifter work", "worker A" will use the same "lifter" that is already in use to perform the "lifter work" separately, thus not ensuring consistency. Therefore, in situations like those illustrated above, the server device 100 restricts the recording of records of "lifter work" performed by "worker A" using the "lifter" that is to be registered. Specifically, the server device 100 restricts the recording of records to be registered if at least a portion of the work period overlaps with records already registered, and the person performing the work also overlaps. This ensures consistency between the information to be registered and other information that has already been registered. In the example shown in Figure 6, record V1024, which corresponds to "Worker C" designated as the subject, is an example of a "first record." Furthermore, record V1023, which is registered together with record V1024 and corresponds to "Lifter," is an example of a "second record."

[0058] With the mechanism described above, for example, even in situations where a worker uses equipment such as a lift to perform a task, it becomes possible to register information regarding the performance of the task specified for either the worker or the equipment for other entities as well. In other words, with the information processing system according to this embodiment, it becomes unnecessary to individually register information related to the management of the status of work performance (e.g., work assignment and the duration of the work) for each of the multiple entities, thereby reducing the burden on the user in terms of such registration. Furthermore, due to these characteristics, it becomes possible to prevent situations in which different information is mistakenly registered for multiple entities that should have the same information registered. That is, it becomes possible to ensure consistency between multiple related records, and it is expected that the burden on the user in terms of checking the consistency between such multiple records will be reduced. In addition, as mentioned above, when new information (records) is to be registered, if consistency is not ensured with already registered information (records), the registration of new information will be restricted. Due to these characteristics, users will no longer need to check each time whether information has been registered incorrectly, and it is expected that the burden on the user will be further reduced.

[0059] (Playback of video) Next, referring to Figure 8, we will explain the functions related to the playback of moving images. When focusing on the operation of a specific imaging device, the imaging results from that device are not necessarily output as a single image file, nor is the imaging device always taking images without interruption. In light of these circumstances, the server device 100 may continuously play back the moving images corresponding to each of the multiple image files resulting from the imaging results of a common imaging device, according to the date and time (timestamp) when the images were taken. Specifically, the server device 100 identifies the date and time each scene of the video was captured based on information such as the timestamp of the image data and the playback time of the video corresponding to the image data. Then, the server device 100 may play back the video corresponding to each of the multiple image data obtained from a common imaging device in chronological order according to the date and time each scene of the video was captured.

[0060] For example, in the example shown in Figure 8, the image data corresponding to the imaging results from each of the two imaging devices is grouped into "Video Group A" and "Video Group B" for each imaging device. Then, the server device 100 plays back the video images corresponding to each of the multiple image data belonging to each group, in chronological order according to the date and time each scene of the video was captured. Specifically, the server device 100 may play back the video images corresponding to files A1 to A4 belonging to Video Group A in sequence. Similarly, the server device 100 may play back the video images corresponding to files B1 to B7 belonging to Video Group B in sequence.

[0061] Furthermore, the server device 100 may play back in parallel video footage corresponding to the imaging results from multiple different imaging devices, while synchronizing the date and time each scene of each video footage was captured. The date and time each scene of each video footage was captured can be determined from the timestamp of the image data corresponding to each video footage, or from information such as the playback time of the video footage, as described above. This makes it possible, for example, to display the playback results of two different video footages in a synchronized state in the display areas V104a and V104b of the management screen V100 exemplified in Figure 4. Furthermore, in this case, there may be no image data outside of the shooting time of each imaging device. The server device 100 may take into account the dates and times outside of the shooting time for each imaging device and play back the video corresponding to the imaging results of each imaging device in parallel, after synchronizing the capture of each scene of the video.

[0062] By applying the above-described controls, administrators are no longer required to perform cumbersome tasks such as concatenating image data from each imaging device to play back a series of videos, or synchronizing videos based on the imaging results from multiple imaging devices.

[0063] Furthermore, as mentioned above, it is possible to identify the date and time each scene of a video was captured from information such as the timestamp of the image data and the playback time of the corresponding video. Therefore, this information indicating the date and time each scene of the video was captured can also be used to register information related to the duration of the work (for example, the start date and end date of the work) as explained with reference to Figure 7. As a specific example, in the management screen V100 shown in Figure 4, with the playback results of a video displayed in display areas V104a and V104b, an instruction to register or update information related to the management of the work status by a desired entity (e.g., an operator) is given via the management interface V102. In this case, the server device 100 may automatically input information regarding the work period to be registered or updated (e.g., the start date and time or end date and time of the work) according to the information indicating the date and time when the scene displayed in display areas V104a and V104b was captured. By applying such controls, administrators can reduce the effort required to register information regarding the status of work performed by various entities, such as workers and equipment.

[0064] (Information output) Next, referring to Figure 9, we will explain the functions related to the output of registered information. As explained with reference to Figures 5 to 7, in the information processing system according to this embodiment, records related to the management of work by each entity such as workers and equipment are registered, and these records include at least the entity performing the work (e.g., worker, equipment, etc.), the work performed by that entity, and the period during which the work was performed. Therefore, the server device 100 may have a function to output the series of records registered in this manner as a list. For example, Figure 9 shows an example of how a list of registered records is displayed on the screen. By presenting a list of registered records in this way, it becomes possible to show administrator users a list of tasks that have been registered for management and analysis.

[0065] Furthermore, when presenting a list of registered records, the server device 100 may also present information corresponding to the aggregated and statistical results of the information contained in each record. For example, in the case shown in Figure 9, the server device 100 may aggregate the work implementation periods by each entity based on the information regarding the work implementation period registered in each record, and then present the results of that aggregation.

[0066] Furthermore, the system may be equipped with a function to output registered information not only on the screen but also through other output means. As a specific example, the system may be equipped with a function to output a list of registered records and the information recorded in those records as data in a predetermined format such as CSV (Comma Separated Values). By providing such a function, it becomes possible, for example, to output a list of tasks that have been registered as targets for management or analysis as data, and to use that data as input data for other analytical devices.

[0067] (Image analysis) Next, referring to Figures 10 to 15, we will describe below an example of a mechanism that utilizes the analysis results of images (for example, video) to further reduce the burden on administrators in managing and analyzing the work status of each entity. When information related to the management of the status of various tasks performed by workers, equipment, and other work entities is entered while reviewing video footage corresponding to the imaging results of the work environment, the longer the period subject to management and analysis, the greater the burden of reviewing the video footage becomes. Specifically, as the period subject to management and analysis lengthens, the recording time of the video footage also increases in proportion to the length of that period, and the period during which the manager is tied up to review a series of scenes in the video footage also increases.

[0068] In light of these circumstances, the information processing system according to this embodiment, for example, analyzes images corresponding to the imaging results of the work environment to recognize and determine the subject captured in the image and the work being performed by that subject, and uses the results to reduce the burden on the manager. For the analysis of images (e.g., video), well-known technologies such as machine learning (ML), deep learning (DL), and computer vision (CV) can be used. For convenience, in the following explanation, CV will be assumed to be used for image analysis. A variety of methods can be employed to recognize and determine desired objects in images using cross-validation (CV). Specific examples of such methods include color detection, optical flow, and feature matching, as well as combinations of these methods.

[0069] Here, with reference to Figures 10 to 12, an example of a screen for specifying setting information related to the analysis of an image (for example, a video) and its operation will be explained. For convenience, in the example shown in Figures 10 to 12, the image to be analyzed will be assumed to be a video. First, let's explain Figure 10. Figure 10 shows an example of screen V400, which serves as the starting point for using various image analysis functions. Screen V400 displays a display area V403 for displaying the playback results of the image to be analyzed, and an operation interface V404 for receiving instructions from the administrator regarding the playback of the said image. Screen V400 also has a menu area V401 for receiving instructions from the user regarding the calling of each of the various image analysis functions. Note that screen V400 may be implemented as one form of the management screen V100 shown in Figure 4, or as another screen loaded from the management screen V100. For example, in the example shown in Figure 10, the menu area V401 displays buttons for calling the "Preview," "CV Control," and "CV Batch Execution" functions. The "Preview" function is used to show the administrator a preview of the execution status of the analysis process set for the target image (e.g., a video). The "CV Control" function is used to specify various settings related to image analysis. The "CV Batch Execution" function is used to execute one or more analysis processes set for the target image as a series of batch processes. The "CV control" function will be explained in more detail below. When button 402, which is associated with the "CV control" function, is pressed, the screen shown in Figure 11 is displayed.

[0070] Figure 11 shows an example of screen V410 for receiving instructions from the administrator regarding the addition or update of analysis settings for a target image. Screen V410 displays a list of analysis processes set for the target image, along with setting information related to that analysis process (for example, the period in the video where the analysis process is set, the location and size of the area to be analyzed, etc.). Screen V410 also displays buttons V411 for receiving instructions to add setting information related to the image analysis process, and buttons V412 for receiving instructions to update already added setting information. When button V411 or V412 is pressed, the settings screen shown in Figure 12 is displayed.

[0071] Figure 12 shows an example of a settings screen V420 for receiving setting information from the administrator regarding the analysis process to be performed on the target image. The settings screen V420 includes input areas V421 to V425, a button V426, and a display area V427.

[0072] Display area V427 is the area where the playback results of the video to be analyzed are displayed as a preview. For example, display area V427 may display as a preview the playback results of scenes corresponding to the period specified via input area V421 from the video to be analyzed. Alternatively, display area V427 may superimpose the settings of the area to be analyzed as a preview onto the image. As a specific example, information indicating the area to be analyzed may be superimposed on the video displayed as a preview in display area V427, according to the information specified via input area V422, which will be described later.

[0073] Input area V421 is an input area for receiving the administrator's specification of the period to be analyzed within the video. In the example shown in Figure 12, the start time and end time of the analysis process can be specified via input area V421 based on the date and time the video was captured. The playback result of the scene corresponding to the date and time specified by input area V421 may be displayed as a preview in display area V427.

[0074] Input area V422 is an input area for receiving the administrator's specification of the position and size of the area to be analyzed within the image (each frame of the video). In the example shown in Figure 12, input area V422 allows the user to specify the coordinates of the top-left and top-right corners of the area to be analyzed, and the rectangular area identified by these coordinates is set in the image as the area to be analyzed. Note that the example shown in Figure 12 is merely an example, and there are no particular limitations on the method of specifying the position and size of the area to be analyzed, as long as it is possible to specify them. Furthermore, the area that can be specified is not limited to a rectangular area; areas of other shapes may also be specified. Note that the method of specifying the position and size of the area may be appropriately changed depending on the shape of the area to be specified.

[0075] Input area V423 is an input area for receiving specifications from the administrator regarding the image analysis method and the judgment method based on the results of the image analysis. As mentioned above, the image analysis method can be specified as a color detection method, optical flow, feature matching, or a combination of these methods. Furthermore, if there are multiple candidate judgment methods for each image analysis method based on the results of the image analysis, it is possible to specify one of the candidate judgment methods. For example, in the example shown in Figure 12, optical flow is specified as the image analysis method, and a judgment method using absolute values ​​is specified as the judgment method. Input area V424 is an input area that accepts from the administrator to specify the threshold applied to the judgment using the results of image analysis, and the value assigned to each state distinguished by that threshold. As a specific example, if optical flow is specified as the image analysis method, the threshold used to judge the difference between frames may be set. In this case, if the difference between frames is less than the threshold, a value indicating that the object being judged for motion is stationary may be set, and if the difference is greater than or equal to the threshold, a value indicating that the object is in motion may be set. Input area V425 is an input area that receives parameter specifications from the administrator to be applied to image analysis in accordance with the specifications in input area V423, and to the judgment based on the results of said image analysis.

[0076] Button V426 is an input interface for receiving instructions from the administrator regarding the reflection of information entered in input areas V421 to V425. When button V426 is pressed, analysis processing is set according to the information specified in each of the input areas V421 to V425.

[0077] Figure 13 shows an example of the application of analysis processing to a target image, illustrating a state in which information indicating the area to be analyzed and information corresponding to the result of a judgment using the results of the image analysis are superimposed on the target image. Specifically, in the example shown in Figure 13, optical flow is specified as the image analysis method, and display information V431 indicating the area to be analyzed is superimposed on image V430. Furthermore, the display mode (e.g., changes in brightness or color) of the display information V431 may be controlled according to the results of the image analysis. As a specific example, in the example shown in Figure 13, the display mode of the display information V431 is controlled so that the area where the movement of an object is detected (in other words, the area where the difference between frames is detected) is emphasized. The image V430 shown in Figure 13 is displayed, for example, in the display area V427 of screen V410 shown in Figure 12, or in the display area V403 of screen V400 shown in Figure 10.

[0078] As described above, the server device 100 can perform analysis on the target image and make various judgments using the results of the analysis, thereby recognizing and determining the state of the subject in the image. The server device 100 can utilize these characteristics to analyze images corresponding to the imaging results of the work environment, thereby recognizing and determining the subject captured in the image and the work being performed by that subject, and the results may be used to support the administrator's work in managing and analyzing the status of work performed by each subject. As a specific example, the server device 100 may detect a predetermined action of a subject such as a worker or equipment captured in the image, and based on the detection result of said action, determine the start timing of a predetermined task performed by said subject. As another example, the server device 100 may detect a change in the state (e.g., color) of a predetermined indicator captured in the image, and based on the detection result of said change in state, recognize the state of the task related to said indicator.

[0079] Here, as an example, with reference to Figures 14 and 15, we will describe an example of a function that supports the management and analysis of the work performed by each entity by the administrator, using the results of various judgments based on the image analysis results.

[0080] First, as Example 1, we will describe the example shown in Figure 14. Figure 14 shows an example of how the status of machine scarf work can be automatically recognized by utilizing the results of image analysis on images corresponding to the imaging results of the monitoring environment of the machine scarf work. Machine scarfing is a process that uses fuel gas and oxygen to thermochemically remove scratches and impurities from the surface of steel billets in order to maintain the high quality of the steel material. The main processes involved are preheating, cutting, and fire extinguishing. In this embodiment, the server device 100 analyzes each image taken from different viewpoints of the environment in which the machine scarf work is performed, and uses the results of the analysis to determine the state of the machine scarf work, thereby recognizing the process of the machine scarf work being performed at any given time.

[0081] In the example shown in Figure 14, moving images corresponding to the imaging results of the machine scarfing operation monitoring environment are used to determine which step of the machine scarfing operation is being performed. Specifically, two monitors are displayed on the screen, and each monitor displays moving images of the machine scarfing operation environment from different viewpoints. Of the two monitors shown in Figure 14, the left monitor displays the result of imaging the situation in which the scarfing machine is thermochemically cutting the surface of the steel billet, projected from below the scarfing machine in a darkened state. In the image displayed on this left monitor, the area to be analyzed is set in the upper part of the screen. The right monitor displays the result of imaging the scarfing machine from a relatively close position across a passageway. In the image displayed on this right monitor, the area to be analyzed is set in the area corresponding to the far end of the passageway (i.e., the area where the scarfing machine is being imaged).

[0082] Screens V441, V443, and V445 schematically show images corresponding to the imaging results of the monitoring environment of the machine scarf operation while the "preheating," "slicing," and "fire extinguishing" processes of the machine scarf operation are being carried out. Display information V442, V444, and V446 schematically show examples of information displayed on the screen according to the results of various judgments using image analysis while the "preheating," "slicing," and "fire extinguishing" processes of the machine scarf operation are being carried out.

[0083] First, referring to screen V441 and display information V442, the process related to the recognition of the "preheating" process will be explained. When the scarfing machine starts preheating, the image displayed on the right monitor shows the entire screen, including the area to be analyzed, glowing white due to the flash of light emitted in conjunction with the preheating by the scarfing machine located at the back of the aisle. At this time, the image displayed on the left monitor shows a white flame near the center, but no flame is visible within the area to be analyzed. The server device 100 recognizes that the process currently being performed is the "preheating" process based on a combination of the results of image analysis targeting areas set for each of the images displayed on the two monitors in screen V441. As a specific example, the server device 100 recognizes that the process currently being performed is the "preheating" process if the brightness of the area set for the image displayed on the left monitor is less than 60, and the brightness of the area set for the image displayed on the right monitor is 200 or more. Furthermore, the server device 100 displays the state determination results based on the image analysis of the regions set for each of the images displayed on the left and right monitors, and the determination result of the process currently being performed based on a combination of these state determination results, as display information V442. Specifically, the state determination result based on the analysis results of the region set for the image displayed on the right monitor is "ignition". In contrast, the state determination result based on the analysis results of the region set for the image displayed on the left monitor is "stopped". Also, as described above, the server device 100 recognizes that the process currently being performed is the "preheating" process based on the combination of these states, and displays the result of this recognition as display information V442.

[0084] Next, referring to screen V443 and display information V444, the process related to the recognition of the "melting" process will be explained. When melting by the scarfing machine begins, the image displayed on the right monitor continues from the start of preheating, and the entire screen, including the area to be analyzed, is illuminated in white by the flash of light emitted in conjunction with the preheating by the scarfing machine. At the same time, in the image displayed on the left monitor, a flash of flame is momentarily reflected in the area to be analyzed, and the entire area is illuminated in white. The server device 100 recognizes that the process being performed is a "melting" process based on a combination of the results of image analysis targeting areas set for each of the images displayed on the two monitors in screen V443. As a specific example, the server device 100 recognizes that the process being performed is a "melting" process if the brightness of the area set for the image displayed on the left monitor is 60 or higher, and the brightness of the area set for the image displayed on the right monitor is 200 or higher. Furthermore, the server device 100 displays the status determination results based on the image analysis of the regions set for each of the images displayed on the left and right monitors, and the status determination results of the process being performed based on a combination of these status determination results, as display information V444. Specifically, the status determination result based on the analysis results of the region set for the image displayed on the right monitor is "ignition". Similarly, the status determination result based on the analysis results of the region set for the image displayed on the left monitor is also "ignition". In addition, as described above, the server device 100 recognizes that the process being performed is the "melting" process based on the combination of these states, and displays the result of this recognition as display information V444.

[0085] Next, referring to screen V445 and display information V446, the process related to the recognition of the "fire extinguishing" process will be explained. When fire extinguishing by the scarfing machine begins, the area set in each image displayed on the right monitor and the left monitor becomes dark (low brightness). The server device 100 recognizes that the process being performed is the "fire extinguishing" process based on a combination of the results of image analysis targeting areas set for each of the images displayed on the two monitors in screen V445. As a specific example, the server device 100 recognizes that the process being performed is the "fire extinguishing" process if the brightness of the area set for the image displayed on the left monitor is less than 60 and the brightness of the area set for the image displayed on the right monitor is less than 150. Furthermore, the server device 100 displays the status determination results based on the image analysis of the regions set for each of the images displayed on the left and right monitors, and the status determination results of the process being executed based on the combination of these status determination results, as display information V446. Specifically, the status determination result based on the analysis results of the region set for the image displayed on the right monitor is "ignition". The status determination result based on the analysis results of the region set for the image displayed on the left monitor is also "stopped". In addition, as described above, the server device 100 recognizes that the process being executed is the "extinguishing" process based on the combination of these states, and displays the result of this recognition as display information V446.

[0086] As described above, the server device 100 can recognize each step of the machine scarf operation by utilizing the results of image analysis on images corresponding to the imaging results of the environment in which the machine scarf operation is performed. As a result, for example, the server device 100 can use the recognition results of each step of the machine scarf operation to automatically input various information regarding the implementation status of that step, thereby reducing the burden on the administrator in inputting such information.

[0087] Next, as Example 2, we will describe the example shown in Figure 15. Figure 15 shows an example of how the status of an automatic polishing machine operation can be automatically recognized by using the results of image analysis on images corresponding to the imaging results of the work environment of the automatic polishing machine operation. In the example shown in Figure 15, the status of the automatic polishing machine operation can be recognized by using the results of image analysis on images corresponding to the imaging results of the "signal lamp," "overhead crane," and "workbench."

[0088] For example, display area V501a displays an image corresponding to the imaging result of the signal lamp, and the area to be analyzed is set to the area corresponding to the blue signal lamp in that image. When a worker enters the equipment's movable area, safety measures are taken before entering the area, and the blue signal lamp sounds at this time. Therefore, for example, by using the results of image analysis on the image corresponding to the imaging result of the signal lamp, it is possible to detect the timing when the blue signal lamp lights up, thereby detecting the time period when a worker is inside the equipment's movable area.

[0089] Furthermore, display area V501b displays an image corresponding to the imaging results of the situation in which the metal roll to be polished is placed on the polishing machine by the overhead crane, and the area to be analyzed for image analysis is set to the area into which the overhead crane enters when the roll is placed. In other words, when the metal roll is placed on the polishing machine, the overhead crane enters the area to be analyzed for image analysis, and after polishing is completed, the overhead crane leaves that area. Therefore, for example, by detecting the period during which the overhead crane that has entered the area to be analyzed for image analysis is stopped within that area, it is possible to detect that period as the polishing time.

[0090] Furthermore, the display area V501c displays an image corresponding to the imaging results of the work environment where the workbench is installed, and the area to be targeted for image analysis is set for the area corresponding to the workbench. Workers enter the area where the workbench is installed when work using that workbench is performed. Therefore, for example, by detecting the entry of workers into the area where the workbench is installed, it becomes possible to detect the period of work using that workbench.

[0091] Furthermore, by combining the results of image analysis on images corresponding to the imaging results of signal lamps, overhead cranes, and workbenches, it is possible to identify the status of automatic polishing machine operations in more detail.

[0092] As described above, the server device 100 can recognize the status of the automatic polishing machine operation by utilizing the results of image analysis on images corresponding to the imaging results of the environment in which the automatic polishing machine operation is performed. As a result, for example, the server device 100 can use the recognition results of the status of the automatic polishing machine operation to automatically input various information related to the status of the operation, thereby reducing the burden on the administrator in inputting such information.

[0093] Furthermore, the results of determining various states based on the analysis of images (e.g., video) can be applied to purposes other than managing the status of work performed by each entity. For example, by determining various states related to the performance of work based on the analysis results of images taken from the work environment, the results of this determination can be used to monitor the status of the work being performed. As a specific example, if a worker enters an area where safety measures are required before entry without taking those safety measures, it is possible to detect the worker's entry into that area and execute a process such as issuing an alert.

[0094] <Conclusion> As described above, the information processing device according to this embodiment receives the designation of multiple entities involved in the execution of the work to be managed, and the designation of the work to be assigned to the multiple entities, and acquires information regarding the execution period of the designated work. Then, for each of the multiple entities, the information processing device records a record related to the management of the work of each of the multiple entities by associating the designated work and the execution period of the work for which information has been acquired as common information.

[0095] This configuration eliminates the need to individually register records related to the status of work completion for each of the multiple entities, thereby reducing the burden on users involved in such registration. Furthermore, this characteristic prevents situations where incorrect information is mistakenly registered for multiple entities that should have the same information. In other words, it becomes possible to ensure consistency between multiple related records, and it is expected that the burden on users involved in verifying the consistency between such records will also be reduced.

[0096] It should be noted that the embodiments described above are merely examples and do not necessarily limit the configuration or processing of the present invention. Various modifications and changes may be made without departing from the technical concept of the present invention. Furthermore, the present invention includes a program that realizes the functions of the embodiments described above, and a recording medium that can be read by a computer storing the program. [Explanation of symbols]

[0097] 1. Information Processing System 100 Server Devices 101 Communications Department 102 Input / Output Control Unit 103 Input Analysis Unit 104 Regeneration Control Unit 105 Image Analysis Department 106 Management Department 107 Storage section 200 terminal devices 300 Imaging devices

Claims

1. A means for receiving the designation of multiple entities from a series of entities under management to be synchronized for the execution of managed work, and the designation of work to be assigned in common to those multiple entities, The means for obtaining a first timestamp indicating the start date and time of the work and a second timestamp indicating the end date and time of the work are provided for the specified work period. A recording means for generating a record for each of the specified multiple entities, including entity identification information, work identification information, a first timestamp, and a second timestamp as items, by synchronizing the work identification information, the first timestamp, and the second timestamp, and for recording the records generated for each of the multiple entities collectively in a storage unit. Equipped with, The recording means does not record the record if at least a portion of the implementation period specified by the first and second timestamps associated with the record to be registered overlaps with at least a portion of the implementation period associated with other records already recorded, and the entity associated with the record overlaps with the entity associated with the other records. At least one of the first timestamp or the second timestamp is obtained by detecting the action or state of at least one of the multiple entities from a video image of the work environment in which the specified work is being performed, and identifying the timing indicating the start or end of the action or state based on the detection result. Information processing device.

2. The aforementioned receiving means accepts the designation of workers who are to be assigned to the work for which the designation is accepted, as at least one of the multiple entities. The information processing apparatus according to claim 1.

3. The aforementioned receiving means accepts the designation of the workers to be assigned the work for which the designation is accepted, and the equipment to be used by said workers, as multiple entities. The information processing apparatus according to claim 2.

4. The receiving means receives the designation of the worker, the task to be assigned to the worker, and the equipment to be used by the worker in the task. The recording means is Information relating the designated worker, the designated task, and the acquired execution period of the task is recorded as a first record corresponding to the worker. For the equipment designated as the target to be used by the worker, information relating the work and the period of implementation recorded in the first record corresponding to the worker is recorded as a second record corresponding to the equipment. The information processing apparatus according to claim 3.

5. The acquisition means acquires at least one of the first timestamp and the second timestamp based on a user's specification. The information processing apparatus according to any one of claims 1 to 4.

6. The aforementioned receiving means accepts the designation of each of the multiple entities from a list of candidates that has been registered in advance for each type of entity. The information processing apparatus according to any one of claims 1 to 5.

7. An information processing method performed by an information processing device, A receiving step that accepts the designation of multiple entities from a series of entities under management to be included in the synchronization of the execution of the managed work, and the designation of work to be assigned in common to those multiple entities, The acquisition step includes obtaining a first timestamp indicating the start date and time of the work and a second timestamp indicating the end date and time of the work as the period for performing the specified work, A recording step of generating a record for each of the specified multiple entities, including entity identification information, work identification information, a first timestamp, and a second timestamp as items, by synchronizing the work identification information, the first timestamp, and the second timestamp, and then recording the records generated for each of the multiple entities collectively in a storage unit. Includes, The recording step does not record the record if at least a portion of the implementation period specified by the first and second timestamps associated with the record to be registered overlaps with at least a portion of the implementation period associated with other records that have already been recorded, and the entity associated with the record overlaps with the entity associated with the other records. At least one of the first timestamp or the second timestamp is obtained by detecting the action or state of at least one of the multiple entities from a video image of the work environment in which the specified work is being performed, and identifying the timing indicating the start or end of the action or state based on the detection result. Information processing methods.

8. On the computer, A receiving step that accepts the designation of multiple entities from a series of entities under management to be included in the synchronization of the execution of the managed work, and the designation of work to be assigned in common to those multiple entities, The acquisition step includes obtaining a first timestamp indicating the start date and time of the work and a second timestamp indicating the end date and time of the work as the period for performing the specified work, A recording step of generating a record for each of the specified multiple entities, including entity identification information, work identification information, a first timestamp, and a second timestamp as items, by synchronizing the work identification information, the first timestamp, and the second timestamp, and then recording the records generated for each of the multiple entities collectively in a storage unit. Make it run, The recording step does not record the record if at least a portion of the implementation period specified by the first and second timestamps associated with the record to be registered overlaps with at least a portion of the implementation period associated with other records that have already been recorded, and the entity associated with the record overlaps with the entity associated with the other records. At least one of the first timestamp or the second timestamp is obtained by detecting the action or state of at least one of the multiple entities from a video image of the work environment in which the specified work is being performed, and identifying the timing indicating the start or end of the action or state based on the detection result. program.

Images