Information processing system and information processing method

Abstract

An information processing system for executing a simulation comprises: a work process analysis part (5) that generates work decomposition data related to a plurality of work processes, on the basis of work site data related to the plurality of work processes and work recognition data acquired by recognizing one or more work units repeated in each of the work processes from a video; and a work simulation part (6) that executes a simulation related to the work processes, on the basis of the work decomposition data.

Description

Information Processing System and Information Processing Method

[0001] The present disclosure relates to an information processing system and an information processing method that execute a simulation on a work process based on data obtained from video captured of the work status of a worker.

[0002] Technologies for executing a simulation on a target work based on data obtained from video capturing a work site at a manufacturing or logistics base or the like have been developed. A work manager can improve work efficiency by reviewing staffing plans for workers based on the results of the simulation.

[0003] Conventionally, with respect to a technology for formulating worker placement plans, a staffing planning device is known that virtualizes a line in a distribution center using a line simulator based on information necessary for executing a simulation input by a manager or the like (for example, the total number of workers and daily order information), and formulates a staffing plan for a picking process, an inspection process, and a packing process every predetermined time (for example, 30 minutes) (Patent Document 1).

[0004] Japanese Patent No. 6841339

[0005] The prior art disclosed in the above Patent Document 1 aims to formulate a plurality of worker placement plans for each work process implemented in a distribution center without imposing a processing load on the device.

[0006] However, in the above prior art, the information input for executing the simulation is only basic information such as the total number of workers and daily order information, so it is difficult to execute the simulation with high accuracy and stably.

[0007] Therefore, the main object of the present disclosure is to provide an information processing system and an information processing method that can improve simulation accuracy when executing a simulation on a work process based on data obtained from video captured of the work status of a worker.

[0008] The information processing system disclosed herein is an information processing system that performs a simulation of a work process based on data obtained from video footage of a worker's work, and comprises: a work process analysis unit that generates work breakdown data for a plurality of work processes based on work site data for a plurality of work processes and work recognition data obtained by recognizing one or more work units repeated in each work process from the video footage; and a work simulation unit that performs a simulation of the work process based on the work breakdown data.

[0009] Furthermore, the information processing method disclosed herein is an information processing method by an information processing system that performs a simulation of work efficiency based on data obtained from video footage of a worker's work, and is configured to generate work breakdown data for a plurality of work processes based on work site data for a plurality of work processes and work recognition data obtained by recognizing one or more work units repeated in each work process from the video footage, and to perform a simulation for the work processes based on the work breakdown data.

[0010] According to this disclosure, when performing a simulation of a work process based on data obtained from video footage of workers performing their tasks, the accuracy of the simulation can be improved.

[0011] Figure 1 shows the overall configuration of the information processing system according to this embodiment. Figure 1 shows an explanatory diagram illustrating an example of multiple work processes to be analyzed by the work process analysis server. Figure 1 shows the hardware configuration of the work process analysis server. Figure 1 shows the functional block diagram of the work process analysis server and the work simulation server. Figure 1 shows an explanatory diagram illustrating an example of work log data. Figure 1 shows an explanatory diagram illustrating an example of shipment count data. Figure 1 shows an example of work breakdown data (before data cleansing). Figure 2 shows an explanatory diagram regarding data cleansing processing of work breakdown data. Figure 3 shows an example of work breakdown data (after data cleansing). Figure 4 shows an example of standard time data. Figure 4 shows a modified version of the functional block diagram. Figure 4 shows an example of simulation data. Figure 4 shows an example of data acquisition setting screen. Figure 5 shows an example of simulation screen. Figure 6 shows an example of simulation screen. Figure 6 shows an example of simulation screen. Figure 7 shows the flow of work breakdown data generation processing by the work process analysis server. Figure 8 shows the flow of data cleansing processing of work breakdown data by the work process analysis server. Figure 9 shows the flow of standard time calculation processing by the work process analysis server. Figure 1 shows the flow of data transmission processing by the work process analysis server.

[0012] The first invention made to solve the aforementioned problems is an information processing system that performs a simulation of a work process based on data obtained from video footage of an worker's work, comprising: a work process analysis unit that generates work breakdown data for a plurality of work processes based on work site data for a plurality of work processes and work recognition data obtained by recognizing one or more work units repeated in each work process from the video footage; and a work simulation unit that performs a simulation of the work process based on the work breakdown data.

[0013] According to this, when performing a simulation of a work process based on data obtained from video footage of workers performing their tasks, the data related to one or more work units recognized from the video is reflected in the simulation, thereby improving the accuracy of the simulation.

[0014] Furthermore, the second invention is configured such that the work process analysis unit or the work simulation unit performs data cleansing on the work breakdown data, and the work simulation unit performs the simulation based on the work breakdown data on which the data cleansing has been performed.

[0015] According to this, since the simulation is performed based on data-cleansed work-decomposition data, the accuracy of the simulation can be effectively improved.

[0016] Furthermore, the third invention further comprises a data cleansing unit that performs data cleansing processing on the work breakdown data, and the work simulation unit is configured to perform the simulation based on the work breakdown data on which the data cleansing processing has been performed.

[0017] According to this, since the simulation is performed based on data-cleansed work-decomposition data, the accuracy of the simulation can be effectively improved.

[0018] Furthermore, the fourth invention is configured such that the work breakdown data includes the work time for each of the work units, and in the data cleansing process, if the work time for a work unit falls outside its appropriate range, the data relating to the work process including that work unit is excluded from the work breakdown data used in the simulation.

[0019] According to this, data related to inappropriate work processes (in this case, work processes that include work units whose work time falls outside the appropriate range) are excluded from the simulation, thus effectively improving the accuracy of the simulation.

[0020] Furthermore, the fifth invention is configured such that, in the data cleansing process, if at least one of the work recognition data and the work site data cannot be successfully acquired with respect to the work process, the data related to that work process is excluded from the work breakdown data used in the simulation.

[0021] According to this, data related to inappropriate work processes (in this case, work processes in which at least one of work recognition data and work site data cannot be successfully acquired) is excluded from the simulation, thereby effectively improving the accuracy of the simulation.

[0022] Furthermore, the sixth invention is configured such that, in the data cleansing process, if the work site data relating to the work process cannot be obtained, at least a portion of the work site data is supplemented by the work recognition data.

[0023] According to this, the lack of on-site work data is compensated for by work recognition data, allowing for stable simulation execution.

[0024] Furthermore, the seventh invention is configured such that the work simulation unit displays a setting screen for the user to input a setting value for the target period related to the generation of the work breakdown data.

[0025] According to this, the appropriate target period for generating work breakdown data can be set by the user, thereby effectively improving the accuracy of the simulation.

[0026] Furthermore, the eighth invention is an information processing method by an information processing system that performs a simulation of work efficiency based on data obtained from video footage of an worker's work, wherein the system generates work breakdown data for a plurality of work processes based on work site data for a plurality of work processes and work recognition data obtained by recognizing one or more work units repeated in each work process from the video footage, and performs a simulation of the work processes based on the work breakdown data.

[0027] According to this, when performing a simulation of a work process based on data obtained from video footage of workers performing their tasks, the data related to one or more work units recognized from the video is reflected in the simulation, thereby improving the accuracy of the simulation.

[0028] The embodiments of this disclosure will be described below with reference to the drawings.

[0029] The information processing system 1 shown in Figure 1 performs simulations of work processes based on data obtained from video footage of workers performing their tasks. The information processing system 1 includes a camera 2, a recorder 3, a management system 4, a work process analysis server (an example of a work process analysis unit) 5, a work simulation server (an example of a work simulation unit) 6, an artificial intelligence (AI) server 7, and a user terminal 8. These components of the information processing system 1 can communicate with each other via a known communication network 10, such as the Internet or a dedicated line network, as needed.

[0030] Camera 2 is an omnidirectional camera or box camera that is placed in a workplace such as a manufacturing or logistics base to film workers performing their tasks and generate video footage of the work. Although only one camera 2 is shown in Figure 1 for convenience, typically, one or more cameras 2 may be installed in the information processing system 1 to correspond to each work process (see Figure 2).

[0031] Recorder 3 records the video output from camera 2. However, in the information processing system 1, recorder 3 is not essential, and the video generated by camera 2 may be stored in camera 2 itself or in other recording devices.

[0032] Management system 4 includes information processing devices such as PCs and servers that execute processes to manage the work of multiple workers at the target facility (for example, manufacturing equipment or a logistics warehouse). When management system 4 manages work in a logistics warehouse, for example, it can implement the functions of a Warehouse Management System (WMS).

[0033] Furthermore, the management system 4 generates a work log 12 (an example of work site data) as information indicating the work status (or work results) of the target equipment, and sequentially stores it in a storage device such as a storage device. The work log 12 includes, for example, the start time and end time of each work process. The work log 12 may also include information on the operating status of the hardware at the work site, information on the equipment such as communication history, etc.

[0034] The work process analysis server (hereinafter referred to as the "analysis server") 5 acquires video footage related to each work process from the camera 2 or recorder 3, and performs analysis on each work process based on that video footage.

[0035] The work simulation server (hereinafter referred to as the "simulation server") 6 obtains analysis results from the analysis server 5 and executes a simulation related to the target work process based on those analysis results. The simulation server 6 can, for example, execute simulations related to work time and work costs (e.g., labor costs).

[0036] The generative AI server 7 performs dialogue processing using a known generative model constructed by deep learning with a large dataset. Known large-scale language models (LLMs) and large-scale vision language models (VLMs) are used as generative models. This enables natural language analysis and generation, as well as integrated analysis and generation of images and language. In this embodiment, the generative AI server 7 performs processing such as analyzing the causes of abnormal work and proposing improvements related to abnormal work, in response to requests from the analysis server 5.

[0037] User terminals 8 are used by users such as project managers and the workers themselves, and consist of PCs, tablet devices, and smartphones. User terminals 8 are used for various settings, inputting operation instructions, and viewing information in the information processing system 1. User terminals 8 may be set up for multiple users.

[0038] Furthermore, in the information processing system 1, at least some of the camera 2, recorder 3, management system 4, generation AI server 7, and user terminal 8 are not essential components of the information processing system 1. In other words, at least some of them may be used as external devices or systems that are not included in the information processing system 1 (i.e., they cooperate with the information processing system 1).

[0039] Figure 2 shows an example of the types of work processes in a logistics warehouse that are analyzed by the analysis server 5. These types of work processes include (A) inspection of goods received from manufacturing plants, etc., (B) picking of goods, (C) packaging of goods, (D) packing of goods for shipment, and (E) shipment of goods to customers, etc.

[0040] In each work process (A)-(E), the same work is repeatedly performed by each worker on the items to be worked on (in this case, products) as they are generated sequentially. The work status of each worker W is captured by a camera 2 that is installed in a corresponding location.

[0041] Next, the hardware configuration of the analysis server 5 will be described with reference to Figure 3.

[0042] The analysis server 5 can be composed of a computer having a known hardware configuration. For example, the analysis server 5 includes a processor 15, memory 16, storage 17, and a network interface 18.

[0043] The processor 15 includes a CPU, GPU, etc., and performs processing to analyze each work process based on a predetermined control program. The memory 16 includes RAM used as a temporary data storage area. The storage 17 includes an SSD or HDD for long-term data storage. The network interface 18 is connected to the communication network 10 and performs data transmission and reception and communication with other devices.

[0044] Furthermore, the functions of the analysis server 5 may be implemented by a distributed processing system or a virtual server. In this case, the analysis server 5 may be configured by a distributed system including, for example, servers, data centers, storage devices, and network devices respectively connected via a communication network. Note that in the present disclosure, the term "apparatus" is not limited to a single device, but is a concept encompassing a configuration in which a plurality of devices function in cooperation with each other.

[0045] Furthermore, except for the difference in their functions, the simulation server 6 and the generative AI server 7 can also employ the same configuration as that of the analysis server 5 described above.

[0046] Next, with reference to FIG. 4, the functions of the analysis server 5 for implementing the information processing method by the information processing system 1 will be described.

[0047] The analysis server 5 has the functions of log data acquisition 21, video acquisition 22, work decomposition 23, abnormality detection 24, factor analysis and improvement proposal 25, work process DB (database) 26, data cleansing 27, and standard time calculation 28. At least a part of these functions is implemented by the processor 15 executing a control program in the configuration of the analysis server 5 as described above.

[0048] In log data acquisition 21, data of the work log 12 is acquired from the management system 4. For example, the analysis server 5 can acquire the accumulated data of the work log 12 at the timing when the data of the work log 12 for a predetermined period (for example, one day) is accumulated in the management system 4. The acquired data of the work log 12 is stored in the work process DB 26 on the storage 17.

[0049] As shown in FIG. 5, for example, the data of the work log 12 includes a management ID, a cycle start time, a cycle end time, and the like. Furthermore, as shown in FIG. 6, for example, the data of the work log 12 may also include the number of product shipments per day and the like.

[0050] The management ID is information used to identify a single work process (for example, a single picking operation shown in Figure 2(B)). The cycle start time and cycle end time are the disclosure time and end time of the single work process, respectively.

[0051] In the video acquisition process 22, video related to each work process is sequentially acquired from camera 2 or recorder 3. The acquired video data is stored in the work process DB 26.

[0052] In task decomposition 23, processing is performed using the work log and video related to the work process stored in the work process DB 26. First, in task decomposition 23, one or more cyclical tasks (an example of a work unit) that are repeated in each work process are recognized based on the video related to the work process. In other words, the analysis server 5 can decompose (i.e., subdivide) each work process into one or more work units. At this time, the analysis server 5 can detect cyclical tasks by performing interaction recognition processing on the video. The analysis server 5 can also obtain the time required for each cyclical task.

[0053] For example, if the work process being processed is a picking operation (see Figure 2(B)), multiple cycle operations may be recognized, such as moving the picking cart to the target shelf, picking up the items stored on the shelf, and moving the picking cart loaded with items to the location of the next process, the packaging operation. However, depending on the type of work process, only one cycle operation may be recognized from the work process.

[0054] Furthermore, in the work breakdown 23, analysis result data (hereinafter referred to as "work breakdown data") is generated based on the data related to each recognized cycle task and the corresponding work log data. The generated work breakdown data is stored in the work process DB 26.

[0055] The work breakdown data includes, for example, the task type, task number, number of cycle classifications, and cycle time, as shown in Figure 7.

[0056] The task type is information about the type of work process (for example, identification information for one of the work processes (A)-(E) shown in Figure 2). The task number is the sequential number of the target work process in a series of similar work processes that are repeatedly performed. The cycle classification number is the number of cycle tasks included in each work process. The cycle time is the time required for each cycle task (an example of work time for each work unit). For example, if the number of cycle tasks is 3, the cycle time includes three times corresponding to those three tasks (for example, 24 seconds, 36 seconds, and 57 seconds). The cycle classification number and cycle time are data obtained from video related to the work process (an example of work recognition data).

[0057] In the work breakdown data shown in Figure 7, the management ID, cycle start time, and cycle end time are obtained from the work log 12 described above. In other words, the data generated by the work breakdown 23 is associated with the data from the work log 12 related to the same work process (in this case, the management ID, cycle start time, and cycle end time). The presence or absence of an anomaly and the anomaly classification are information added by the anomaly detection 24, which will be explained next. Furthermore, the exclusion flag is information added by the data cleansing 27, which will be described later.

[0058] In the anomaly detection 24, abnormal work in the target work process is detected based on the work breakdown data obtained by the work breakdown 23 (data excluding the presence or absence of anomalies, anomaly classification, and exclusion flag shown in Figure 7). Abnormal work includes, for example, work where the time taken for each work unit falls outside the acceptable range. For example, the analysis server 5 can detect cycle work with incorrect work procedures or cycle work that deviates from safety standards by performing known behavior recognition processing on the video.

[0059] Furthermore, in the anomaly detection 24, based on video footage related to the work process, tasks performed using incorrect work procedures or tasks that deviate from safety standards may be detected as abnormal tasks. The detected abnormal tasks are associated with one of several pre-set anomaly classifications (i.e., classified). For example, the analysis server 5 can detect cyclical tasks performed using incorrect work procedures or cyclical tasks that deviate from safety standards by executing known behavior recognition processing on the video footage.

[0060] The analysis server 5 can detect abnormal operations for each cycle operation using the anomaly detection 24 and classify them.

[0061] The data obtained by the anomaly detection 24 (in this case, the presence or absence of an anomaly and the anomaly classification) is added to the work breakdown data in the work process DB 26. In other words, the work breakdown data generated by the work breakdown 23 described above is updated based on the results of the anomaly detection 24.

[0062] In the work breakdown data shown in Figure 7, the presence or absence of an abnormality indicates whether there is an abnormal operation in the target work process ("1" or "0"). The abnormality classification is the identification information for the abnormal operation.

[0063] In Factor Analysis and Improvement Proposal 25, based on the detection results of abnormal work, the cause of the abnormality is analyzed, and improvement proposals for work to prevent the abnormality from occurring are generated.

[0064] The analysis server 5 can send (i.e., input) the detection result of the abnormal work and a prompt instructing the generation AI server 7 to analyze its cause in order to perform an analysis of the cause of the anomaly. At this time, the analysis server 5 can send the video of the corresponding work along with the detection result of the abnormal work. In response, the generation AI server 7 generates the result of the cause analysis (text) and sends (i.e., outputs) it to the analysis server 5.

[0065] Furthermore, the analysis server 5 can send a prompt to the generation AI server 7 instructing it to propose improvements to the work based on the results of its factor analysis, in order to generate improvement plans. In response, the generation AI server 7 generates improvement plans for the abnormal work and sends them to the analysis server 5. Note that the instruction to the generation AI server 7 to propose improvements to the work may be sent together with the instruction for the factor analysis described above.

[0066] Although not illustrated here, the data obtained from the factor analysis and improvement proposal 25 is added to the work decomposition data in the work process DB 26 as data related to the corresponding work process. Note that in the factor analysis and improvement proposal 25, the improvement proposal may be omitted and only the factor analysis may be performed.

[0067] In data cleansing 27 (an example of the data cleansing unit), data cleansing is performed on the work breakdown data in the work process DB 26. In the data cleansing process, data unsuitable for simulation is excluded from the work breakdown data provided to the simulation server 6. Here, in order to improve the accuracy of simulations of future work results, it is necessary to retain abnormal data within the appropriate range as simulation data. However, accidental abnormal data outside the appropriate range should be excluded as unsuitable data because retaining it would lead to a deterioration in simulation accuracy. In other words, useful simulation results can be obtained by repeatedly adjusting the setting of the appropriate range and excluding unsuitable data.

[0068] For example, inappropriate data may include data with cycle times above a threshold or data with cycle times below a threshold (see the data labeled (A) and (B) in Figure 7, respectively). Inappropriate data may also include data with cycle times outside the appropriate range (for example, values ​​that should not exist). In addition, inappropriate data may include data where the results of sensing related to the work have not been detected (for example, data where the video could not be acquired properly) (see the data labeled (C) in Figure 7). In addition, inappropriate data may also include data where the work log 12 has not been detected (for example, data where at least a portion of the work log 12 could not be acquired properly).

[0069] The work decomposition data after data cleansing is given an exclusion flag to identify data to be excluded (see the data labeled (D) in Figure 8), as shown in Figure 8. The exclusion flag indicates whether the corresponding data should be excluded from the data used in the simulation ("1") or not ("0").

[0070] Each work process that makes up the work breakdown data described above includes data based on the work log 12. However, there are work processes for which a work log 12 is not generated even under normal circumstances. In this case, the data cleansing 27 can supplement the data for work processes for which work logs cannot be obtained with data obtained from video footage related to the work process.

[0071] The supplemented data is added as data for one work process that constitutes the work breakdown data, for example, as shown by the symbol (E) in Figure 8. In the data symbol (E) in Figure 8, the cycle start time and cycle end time, which cannot be obtained from the work log 12, are supplemented from the video related to the work process. However, this process of supplementing work log data from video may be omitted.

[0072] In the standard time calculation 28, a standard value (hereinafter referred to as "standard time") is calculated for the time required for each type of work process performed within a predetermined period. In the standard time calculation process, as shown in Figure 9 for example, the work breakdown data corrected by data cleansing 27 (hereinafter referred to as "corrected data") is processed. In the corrected data, inappropriate data is excluded from the simulation, and data supplemented from video is added to the simulation.

[0073] The standard time data obtained through the standard time calculation process is stored in the work process DB26. The standard time data includes, for example, as shown in Figure 10, the standard time for various work processes executed over a predetermined period (in this case, one day), in addition to the standard time mentioned above, the anomaly occurrence rate, the duration of the anomaly, the anomaly classification, and the anomaly occurrence rate (breakdown) for each classification.

[0074] The abnormality rate is the ratio of the number of work processes in which abnormal work was detected to the total number of work processes performed during a given period for a given type of work process. The abnormality duration is the sum of the time during which abnormal work was detected in each work process performed during a given period for a given type of work process. The abnormality classification is the identification information for all abnormal work detected in each work process performed during a given period for a given type of work process. The abnormality rate for each classification is the abnormality rate for that specific abnormality classification.

[0075] In Figure 4, an example is shown in which the analysis server 5 has a work process DB 26 and performs data cleansing 27 in that database. However, the system is not limited to this, and a similar data cleansing process can also be performed on the simulation server 6. In that case, the simulation server 6 can obtain the data used for data cleansing from the analysis server 5. Furthermore, the simulation server 6 may perform the same processing as the standard time calculation 28 described above on the corrected data obtained by the data cleansing process.

[0076] Furthermore, as shown in Figure 11, for example, the functions of the above-described work process DB 26 can also be implemented by a database server 126 (an example of a data cleansing unit) that is provided separately from the analysis server 5. The database server 126 can perform the same processing as the data cleansing 27 described above. The database server 126 is connected to the communication network 10 and effectively acts as an intermediary between the analysis server 5 and the simulation server 6.

[0077] Next, referring again to Figure 4, we will explain the functions of the simulation server 6 for realizing the information processing method by the information processing system 1.

[0078] The simulation server 6 has the functions of data acquisition 31, simulation 32, and drawing 33. At least some of these functions are realized by the processor executing a control program in the configuration of the simulation server 6 as described above.

[0079] In data acquisition 31, simulation data is requested from the analysis server 5, and the data necessary for the simulation is acquired from the analysis server 5. In response to the request from the simulation server 6, the analysis server 5 performs the analysis on each work process as described above and sends the analysis results to the simulation server 6 as simulation data.

[0080] The simulation data includes, for example, task type, number of tasks, start date, end date, standard time, anomaly rate, anomaly duration, anomaly classification, and anomaly rate for each classification, as shown in Figure 12. The number of tasks is the number of times each type of work process is executed during the set data acquisition period. The start date and end date indicate the beginning and end of the data acquisition period. The standard time, anomaly rate, anomaly duration, anomaly classification, and anomaly rate for each classification are data obtained from the standard time calculation process described above for the data during the specified data acquisition period.

[0081] By accessing the simulation server 6 from the user terminal 8, the user can, for example, configure settings related to the data acquisition period, etc., on the data acquisition settings screen 41 shown in Figure 13.

[0082] The data acquisition settings screen 41 allows you to set the data acquisition period (an example of a setting value for the target period), the acquisition target time, the threshold target, and the maximum and minimum values ​​of the threshold. The data acquisition period is the period (in this case, the number of days) during which data used to generate simulation data (i.e., data such as work logs 12 and video related to work processes) is acquired. The acquisition target time is the time period during which data used to generate simulation data is acquired. By setting the acquisition target time appropriately, for example, only data related to valid time periods (e.g., the time period during which the target work is performed) will be subject to analysis by the analysis server 5. The maximum and minimum values ​​are the thresholds (i.e., appropriate ranges) for time-related data. Data above the maximum value and data below the minimum value are excluded from the data used to generate simulation data. The threshold target is the data item to which these maximum and minimum values ​​apply. "all" for the threshold target indicates that the maximum and minimum values ​​apply to all time-related data.

[0083] Users of the simulation server 6 can use the setting values ​​they have configured on the data acquisition settings screen 41 as default values. Alternatively, users may input these settings values ​​when running the simulation. Furthermore, a setting value saving function (not shown) allows users to read and write their configured setting values.

[0084] In Simulation 32, the simulation is executed based on simulation data acquired from the analysis server 5. The simulation results are obtained as numerical values ​​and graphs. Furthermore, by performing a detailed analysis in the simulation, the user can understand the causes and trends of abnormal work and derive directions for improving and optimizing the work process.

[0085] In drawing 33, the simulation results and the results of their detailed analysis are output to the simulation screen. For example, the project manager can use the user terminal 8 to access the dashboard for the simulation server 6 and display the simulation screen on the display.

[0086] For example, as shown in Figure 14, the simulation screen includes a simulation result display field 51 and a simulation condition setting field 54.

[0087] The results display field 51 includes a simulation results tab 52 and a detailed analysis tab 53. By clicking the simulation results tab 52, the user can display the simulation results, including numerical data and graphs. The user can also display the detailed analysis results by clicking the detailed analysis tab 53.

[0088] The detailed analysis results include, for example, an overview view 61 of the equipment being simulated (in this case, a distribution warehouse), images (or videos) 62 of the work processes being analyzed, a summary of the analysis 63, and simulation data 64 (e.g., numerical values ​​or graphs). In the overview view 61, the areas of the work processes being analyzed may be highlighted (highlighted, blinking, text displayed, bordered, etc.). In addition, the simulation results can be played back as an animated video in the overview view 61, allowing users who watch the video to grasp an overview of the overall progress of the work, including tasks performed in parallel.

[0089] The user can input various conditions for performing a detailed analysis in the condition setting field 54. Figure 14 shows an example where input fields 66 for the number of picking workers and 67 for abnormalities are provided in the condition setting field 54. When the user enters the desired conditions and presses the execute button 68, the detailed analysis is performed and simulated data 64 corresponding to those conditions is displayed.

[0090] Figure 14 shows initial simulation data 64 regarding the truck departure time (i.e., the shipping time of the target product). Here, the input field 66 for the number of picking workers has the default value of "5" entered. This indicates that the actual number of picking workers at present is 5. Also, the input field 67 for abnormalities has the default value of "100%" entered. This indicates that the actual value for abnormal work (in this case, the actual value of the abnormality occurrence rate) at present is set to 100%. Furthermore, the simulation data 64 shows that the truck departure time is "17:45", and that the departure time (corresponding to the actual value) is "+15 min" (i.e., a 15-minute delay) compared to the scheduled departure time (target time).

[0091] Therefore, as shown in Figure 15, for example, the user can enter "80%" in the error input field 67, assuming a 20% reduction in the error rate, and perform a detailed analysis. As a result, the simulated data 64 shows that the truck departure time becomes "17:35" and the departure time is "+5 min" (i.e., a 5-minute delay). In other words, it is shown that the delay in the truck departure time can be mitigated by reducing the error rate by 20%.

[0092] Furthermore, as shown in Figure 16, for example, the user can enter "70%" in the abnormality input field 67, assuming a 30% reduction in the abnormality rate, and perform a detailed analysis. As a result, the simulated data 64 shows that the truck departure time becomes "17:30" and the departure time becomes "+0 min" (i.e., no delay). In other words, it is shown that delays in truck departure times can be eliminated by reducing the abnormality rate by 30%.

[0093] Furthermore, the simulation screen can also display labor costs in the simulation data 64, as shown in Figure 17, for example. Here, the simulation data 64 shows the truck departure time and labor costs under the same conditions as shown in Figure 15 (5 picking workers, 20% reduction in the abnormality rate). Regarding labor costs, based on 5 picking workers (actual value), the increase or decrease in costs is "±\0 / month", and it is shown that a total cost of "\1,000,000 / month" is incurred.

[0094] For example, as shown in Figure 18, the user can enter "4" in the input field 66 for the number of picking workers, assuming a reduction of one picking worker, and then perform a detailed analysis. The simulated data 64 shows that the truck departure time is "17:35" (i.e., the same as when there are 5 picking workers). Furthermore, it is shown that the increase or decrease in labor costs is "-¥200,000 / month", resulting in a total of "¥800,000 / month". In other words, it is shown that reducing the number of picking workers by one reduces labor costs compared to when there are 5 picking workers, and moreover, there is no change in the truck departure time (no adverse effects).

[0095] Furthermore, as shown in Figure 19, for example, the user can enter "3" in the input field 66 for the number of picking workers, assuming a reduction of two picking workers, and then perform a detailed analysis. The simulated data 64 shows that the truck departure time is "17:45". Regarding labor costs, it is shown that the increase or decrease in costs is "-¥400,000 / month", resulting in a total of "¥600,000 / month". In other words, it is shown that reducing the number of picking workers by two reduces labor costs compared to the case with five picking workers, but it also results in a 15-minute delay in the truck departure time (a negative consequence).

[0096] Next, with reference to Figure 20, the process of generating work breakdown data by the analysis server 5 will be explained.

[0097] First, the analysis server 5 acquires video footage related to the work process to be analyzed from the camera 2 or recorder 3 (ST101). At this time, the analysis server 5 can either have already acquired the work log data 12 related to the work process to be analyzed from the management system 4, or acquire it in parallel with the video footage.

[0098] Next, the analysis server 5 performs interaction recognition processing on the video related to the work process (ST102). In the interaction recognition processing, for example, a machine learning model is used to analyze the interaction between the worker and the object (i.e., the worker's movements relative to the object), and the worker's movements and the movement of the object are recognized. As a result, the analysis server 5 can recognize each cycle task included in the work process being analyzed and obtain the time required for each cycle task.

[0099] Subsequently, the analysis server 5 performs work breakdown processing (ST103) based on the data related to each cycle of work and the corresponding work log data. The work breakdown data generated by the work breakdown processing includes data obtained from the work log 12 and data obtained from video related to the work process.

[0100] Next, the analysis server 5 performs anomaly detection processing based on the work breakdown data (ST104). The anomaly detection processing detects abnormal work in the target work process, and information regarding the detection result is added to the work breakdown data.

[0101] Furthermore, based on the detection results of abnormal work, the analysis server 5 performs a cause analysis of the abnormality and generates a plan for improving the work to prevent the abnormality from occurring (ST105). The data related to the cause analysis and improvement plan obtained in step ST105 is added to the work breakdown data.

[0102] The work breakdown data obtained through this series of processes is ultimately registered in the work process DB26 (ST106).

[0103] Next, referring to Figure 21, we will explain the data cleansing process of the work breakdown data performed by the analysis server 5.

[0104] First, the analysis server 5 acquires the work breakdown data generated by the process shown in Figure 20 as the data to be processed (ST201). Next, the analysis server 5 determines whether there is any inappropriate data in the work breakdown data for each work process that is outside the appropriate range (for example, above or below a threshold) (ST202).

[0105] Therefore, for work processes where inappropriate data is determined to exist (Yes in ST202), the data exclusion flag (see Figure 8) is set to "1" (ST203). On the other hand, for work processes where no inappropriate data is determined to exist (No in ST202), the exclusion flag remains unchanged from its default value of "0".

[0106] Next, the analysis server 5 determines whether or not there are any missing values ​​in the data for each work process in the work breakdown data that could not be obtained from the sensing results (in this case, video related to the work process) (ST204).

[0107] Therefore, for data related to work processes where missing values ​​are determined to exist (Yes in ST204), the exclusion flag (see Figure 8) is set to "1" (ST205). On the other hand, for data related to work processes where missing values ​​are determined not to exist (No in ST204), the exclusion flag remains unchanged from its default value of "0".

[0108] Next, the analysis server 5 determines whether or not there is data related to work processes for which work logs have not been obtained regarding the target work breakdown data (ST206).

[0109] Therefore, if it is determined that there is data for which work logs have not been acquired (Yes in ST206), data obtained from the sensing results corresponding to that data is added (ST207). In this way, data related to work processes for which work logs cannot be acquired is supplemented with data obtained from the video.

[0110] The corrected data (cleansed work breakdown data) obtained through this series of processes is finally registered in the work process DB26 (ST208).

[0111] Next, with reference to Figure 22, the standard time calculation process by the analysis server 5 will be explained.

[0112] First, the analysis server 5 acquires the corrected data obtained through the process shown in Figure 21 as the data to be processed (ST301). Next, the analysis server 5 separates normal data (exclusion flag is 0) and abnormal data (exclusion flag is 1) based on the value of the exclusion flag in the corrected data (ST302). As a result, abnormal data (i.e., inappropriate data) is excluded from the standard time calculation process.

[0113] Next, the analysis server 5 calculates the standard time for each type of work process based on the normal data (ST303). Subsequently, the analysis server 5 calculates the time it takes for an anomaly to occur for each type of work process (ST304). After that, the analysis server 5 calculates the anomaly occurrence rate for each type of work process (ST305), and further calculates the anomaly occurrence rate for each anomaly classification (ST306).

[0114] The standard time data obtained through this series of processes is finally registered in the work process DB26 (ST307).

[0115] Next, with reference to Figure 23, the data transmission process by the analysis server 5 will be explained.

[0116] First, the analysis server 5 receives a request for simulation data from the simulation server 6 (the data acquisition settings screen 41, which is set by the user) (ST401).

[0117] Next, the analysis server 5 extracts data related to each work process that is the subject of the request (ST402). Subsequently, the analysis server 5 performs processes such as generating work breakdown data, data cleansing of work breakdown data, and standard time calculation on the extracted data, and then integrates the resulting data (ST403).

[0118] Subsequently, the analysis server 5 transmits the simulation data obtained by integrating the data to the simulation server 6 (ST404).

[0119] Thus, according to the information processing system 1 and its information processing method, work decomposition data related to work processes can be generated based on work log data related to multiple work processes and data acquired by recognizing one or more repeating cycle tasks in each work process from video footage of the work. Based on this work decomposition data, a simulation of the work processes can be performed. This improves the accuracy of the simulation.

[0120] As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments.

[0121] The information processing system and information processing method relating to this disclosure have the effect of improving the accuracy of simulations when performing simulations related to work processes based on data obtained from video footage of workers performing their work, and are useful as information processing systems and information processing methods that perform simulations related to work processes based on data obtained from video footage of workers performing their work.

[0122] 1: Information processing system 2: Camera 3: Recorder 4: Management system 5: Work process analysis server (Work process analysis unit) 6: Work simulation server (Work simulation unit) 7: Generation AI server 8: User terminal 10: Communication network 12: Work log 15: Processor 16: Memory 17: Storage 18: Network I / F 21: Log data acquisition 22: Video acquisition 23: Work breakdown 24: Anomaly detection 25: Summary analysis and improvement proposal 26: Work process DB 27: Data cleansing (Data cleansing unit) 28: Standard time calculation 31: Data acquisition 32: Simulation 33: Drawing 41: Data acquisition setting screen 51: Result display field 52: Simulation result tab 53: Detailed analysis tab 54: Condition setting field 61 :Overview 63 :Summary 64 :Simulated data 126 :Database server W :Worker

Claims

1. An information processing system that performs a simulation of a work process based on data obtained from video footage of a worker's work, comprising: a work process analysis unit that generates work breakdown data for a plurality of work processes based on work site data for a plurality of work processes and work recognition data obtained by recognizing one or more work units repeated in each work process from the video footage; and a work simulation unit that performs a simulation of the work process based on the work breakdown data.

2. The information processing system according to claim 1, wherein the work process analysis unit or the work simulation unit performs a data cleansing process on the work breakdown data, and the work simulation unit performs the simulation based on the work breakdown data on which the data cleansing process has been performed.

3. The information processing system according to claim 1, further comprising a data cleansing unit that performs data cleansing on the work breakdown data, wherein the work simulation unit performs the simulation based on the work breakdown data on which the data cleansing has been performed.

4. The information processing system according to claim 2 or 3, wherein the work breakdown data includes the work time for each of the work units, and in the data cleansing process, if the work time for a work unit falls outside its appropriate range, the data relating to the work process including that work unit is excluded from the work breakdown data used for the simulation.

5. The information processing system according to claim 2 or 3, wherein, in the data cleansing process, if at least one of the work recognition data and the work site data cannot be successfully obtained with respect to the work process, the data related to that work process is excluded from the work breakdown data used in the simulation.

6. The information processing system according to claim 2 or 3, wherein, in the data cleansing process, if the work site data relating to the work process cannot be obtained, at least a portion of the work site data is supplemented by the work recognition data.

7. The information processing system according to claim 1, wherein the work simulation unit displays a setting screen for the user to input a setting value for the target period related to the generation of the work breakdown data.

8. An information processing method by an information processing system that performs a simulation of work efficiency based on data obtained from video footage of workers performing their tasks, comprising: generating work breakdown data for a plurality of work processes based on work site data for a plurality of work processes and work recognition data obtained by recognizing one or more work units repeated in each work process from the video footage; and performing a simulation for the work processes based on the work breakdown data.

Images