surveillance system

Abstract

To provide a monitoring system that allows accurate understanding of the movements of people and objects, that can accurately monitor a manufacturing process, and contribute to the improvement of work efficiency.SOLUTION: A monitoring system that monitors people, objects, or devices in a manufacturing process comprises: a camera that monitors a person, an object, or a device; an analysis unit that analyzes video data obtained by the camera; and a display unit that displays analysis results of the analysis unit. The analysis unit detects a change point in the video data and analyzes the state of the person, the object, and the device based on the change point, detects the change point by comparing a signal included in newly acquired video data with a signal included in machine-learned video data, and detects a position where the change point exists to monitor work error in the manufacturing process.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a monitoring system for monitoring people, things, and machines at a manufacturing site. [Background technology]

[0002] Systems are known that monitor people involved in manufacturing, the products being manufactured, and the machines that carry out the manufacturing process (manufacturing equipment, inspection equipment, packaging equipment, etc.) at manufacturing sites, and further improvements to these systems are being made (see, for example, Patent Documents 1 to 3).

[0003] For example, in manufacturing sites in the assembly industry, multiple parts are aggregated and combined to produce a product through operations (e.g., welding, cutting, assembly, etc.) according to prescribed assembly drawings and work procedures. In the case of complex products, multiple work sites perform their respective tasks, and each unit is combined to create a more complex product. In such operations, when a large number of parts (items) are aggregated, the work procedures, site layout, and part aggregation process become complicated. This can lead to issues such as waiting for tasks, parts backlogs, mistimed part aggregation, and worker placement errors, resulting in reduced assembly efficiency. However, particularly in the assembly industry, where equipment and machinery are rarely used and assembly work is primarily performed by humans, a sensor-based logging environment is not yet in place. For this reason, there is a demand for monitoring systems that can accurately grasp and monitor the status of people and items at manufacturing sites. In other words, there is a need to build a digital twin of the assembly industry and utilize it to improve and monitor management figures.

[0004] However, when quantifying and recording a person's work status, the worker must stop what they are doing and record the work status, or another worker must observe the worker's movements and record them, which is time-consuming and costly.

[0005] Furthermore, with regard to monitoring the status of an object, when the object is inside a machine, it is possible to record and monitor the machine's operations, but when an object is moved from a machine at one work site to a machine at another work site, its status cannot be adequately monitored. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 5416322 [Patent Document 2] Japanese Patent Application Publication No. 2019-139570 [Patent Document 3] Japanese Patent Application Publication No. 2019-16226 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention provides a monitoring system that can accurately grasp the movements of people and things, accurately monitor the manufacturing process, and contribute to improving work efficiency. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention provides a monitoring system for monitoring people, objects, and equipment at a manufacturing site, the monitoring system comprising: a camera for monitoring the people, objects, or equipment; an analysis unit for analyzing video data acquired by the camera; and a display unit for displaying the analysis results by the analysis unit. The analysis unit detects change points in the video data and analyzes the status of the people, objects, or equipment based on the change points. [Effects of the Invention]

[0009] According to the monitoring system of the present invention, it is possible to provide a monitoring system that can accurately monitor the manufacturing process and contribute to improving work efficiency. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing an example of the configuration of a monitoring system 1 according to an embodiment. [Figure 2] 1 is a schematic diagram for explaining the operation of the people monitoring camera 11. FIG. [Figure 3] 1 is a schematic diagram for explaining the operation of the object monitoring camera 12 and the device monitoring camera 13. FIG. [Figure 4] 1 is a schematic diagram for explaining the operation of the object monitoring camera 12 and the device monitoring camera 13. FIG. [Figure 5] 4 is a flowchart illustrating the operation of the monitoring system 1 according to the embodiment. [Figure 6] 10 is an example of a display screen on the display PC 17. [Figure 7] 10 is an example of a display screen on the display PC 17. [Figure 8] 10 is an example of a display screen on the display PC 17. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be designated by the same numerals. Note that the accompanying drawings show embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

[0012] Although the present embodiment has been described in sufficient detail to enable those skilled in the art to implement the present disclosure, it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0013] The following describes an embodiment of the process for assembling products (final products and units as intermediate products) composed of multiple parts. Parts ordered and delivered from the core system or ordering system are brought in and stored in a parts warehouse. Meanwhile, when a manufacturing production number is issued in accordance with the product's manufacturing plan, the parts are transported to the assembly shop in accordance with the manufacturing plan (including assembly drawings and assembly procedures created by the design department) predefined for each production number. In this assembly shop, assembly personnel and necessary equipment and jigs are allocated in accordance with the manufacturing plan. Assembly personnel perform the specified assembly work using the necessary equipment and jigs depending on the case, in accordance with the manufacturing plan.

[0014] After the required assembly work is completed, the product is transported to the next assembly shop. This process is repeated and integrated to complete the final product, which undergoes the required shipping inspection and is then stored in a shipping warehouse. This embodiment provides a means and method for recording the assembly process and work progress as they occur within a factory. Specifically, the identification information of each part, each piece of equipment / jig, and each worker, as well as the start / end times and take-off times, are recorded for each warehouse, serving shelf, and assembly shop. Conventional technology requires the installation of sensors individually for each work task, each shop, and each shop configuration to detect the presence or absence of product parts. This requires the installation location of sensors to be changed whenever the work content or the size and shape of parts change, for example, due to product changes. According to the embodiment described below, the installation location of the camera does not need to be changed as long as the part, jig, or worker is within the camera's imaging range. This significantly reduces the installation cost when introducing a system.

[0015] An example of a monitoring system 1 according to an embodiment will be described with reference to Fig. 1. This monitoring system 1 may be composed of, for example, a people monitoring camera 11, an object monitoring camera 12, a device monitoring camera 13, a transfer computer (PC) 14, a collection and storage server 15, an analysis server 16, and a display computer (PC) 17.

[0016] People monitoring camera 11 is a camera for monitoring the movements of people (workers) performing work at a manufacturing site. People monitoring camera 11 stores information about, for example, people's shapes and movement characteristics in advance, and can be configured to quickly and reliably detect human movements.

[0017] The object monitoring camera 12 is a camera for monitoring the movement of objects manufactured and moving at a manufacturing site. For example, the object monitoring camera 12 may be a camera that captures and identifies identification information (one-dimensional code, two-dimensional code, color chart, etc.) attached to a cart to monitor the movement of the cart on which the object is placed and transported. By attaching different identification information to each of multiple carts, the type of cart can be identified. However, this is merely an example, and the object monitoring camera 12 is not limited to monitoring the movement of carts using identification information. For example, the shape of the object itself may be recognized, or various identification information attached to the object may be identified. Furthermore, instead of recognizing carts, the object monitoring camera 12 may monitor the movement of trays placed on a belt conveyor, for example. If the object monitoring camera 12 is configured to read two-dimensional codes, the object monitoring camera 12 may be equipped with an application for analyzing two-dimensional codes. Of course, the application for analyzing two-dimensional codes may also be stored on the cloud.

[0018] The equipment monitoring camera 13 is a camera for monitoring the operation of various devices (manufacturing devices, inspection devices, packaging devices, etc.) installed at the manufacturing site. The equipment monitoring camera 13 can be different depending on the characteristics of the device being imaged. For example, an equipment monitoring camera 13 for a device that requires human operation can be a camera that images the device's operation panel and monitors the human operation. By imaging the operation panel, it is possible to determine whether the device is operating or stopped.

[0019] Today's facilities and equipment can detect parameters that indicate the state of the equipment using sensors installed inside the equipment. However, if the equipment does not have these sensors, it is necessary to modify the equipment to install these sensors. However, skilled workers who work with the equipment can monitor the process stage of the equipment (machining, assembly, or other processing) by visual inspection and listening to the sounds, and can also monitor the state of the equipment to see if there are any abnormalities. Therefore, the status of the equipment can also be monitored by utilizing TV cameras. Introducing monitoring using TV cameras can eliminate the need for expensive monitoring systems, such as modifications to install sensors and monitoring by skilled workers.

[0020] In the case of an automatic device that does not require human operation, the device monitoring camera 13 for that device can be a camera for monitoring, for example, the input port for items (components) or the movement of moving parts (for example, the blade of a cutting device). Furthermore, the device monitoring camera 13 can also be used to inspect the content of various operations within the device. For example, in a device for connecting harnesses, a harness connection confirmation inspection can be performed, and from the images, it can be determined whether the connections are correct or not, and if not, errors such as incorrect wiring can be detected.

[0021] In this embodiment, cameras 11 to 13 may be provided with an inspection object detection means for detecting an inspection object and a defect detection means for detecting defects, either within the cameras 11 to 13 or in an external controller. The inspection object detection means has a function of detecting the inspection object portion in an image of a non-defective product. As an example, the inspection object detection means may be configured with a deep learning network that has previously learned the inspection object portion. By providing this means, it is possible to avoid falsely detecting as defective a portion that is significantly deformed but not considered abnormal (for example, a component such as a cable).

[0022] Then, by inputting the part detected as the part to be inspected into defect detection means, defects in the object to be inspected can be detected. As an example, this means can be configured by having an autoencoder learn multiple normal parts (images of normal products) in advance. By creating a difference image between the image reconstructed by the autoencoder and the input image, if there is an abnormal part, the image signal at that position will have a large value and will be detected as an abnormality through threshold processing. Detected defect information (defect images, product information, etc.) can be stored in defect information storage means.

[0023] When a new detection target is provided, the normal image of the new detection target can be trained by the inspection target detection means and the defect detection means (autoencoder). This training may be performed using 3D CAD information generated during product design.

[0024] Furthermore, the information (network parameters) learned for a new detection target can be reflected by performing a weighted average with the corresponding parameters of an already trained network. This method also accommodates cases where users do not want to upload information about new detection targets to the cloud. In other words, by sending the trained network parameters shared on the cloud to the user's location and performing a weighted average at that location, it is possible to configure and provide a highly accurate defect detection method and inspection target detection method specific to the user.

[0025] The device monitoring camera 13 can use various lenses (ultra close-up lens, fisheye lens, wide-angle lens, telephoto lens, etc.) depending on the characteristics of the object to be imaged and the distance to the object.

[0026] The progress of assembly work at the assembly process site can be identified and recorded by visual confirmation by an expert or manager with specific skills. From this perspective, this embodiment is configured to calculate the takeoff and landing times to be recorded by performing appropriate data processing, as described below, only on the image information captured by a TV camera using visible light.

[0027] Furthermore, when assembly work is completed at each assembly shop during the assembly process, not only is the completion time recorded, but the start and completion times for each assembly process, as well as any defects in the intermediate products that have been assembled, are also detected and recorded. The sampling rate of this TV camera does not need to be the usual 30 fps or 60 fps, but can be thinned to values ​​such as 1 fps or 0.2 fps, as necessary. Conversely, high-speed imaging such as 120 fps or 2000 fps is also acceptable. The former has the effect of reducing the total amount of data stored, while the latter is necessary when collecting data involving high-speed movement.

[0028] The transfer PC 14 has the function of reading video data (which may include both video and still images) captured by the various cameras 11-13 and transferring it to the collection and storage server 15. The transfer PC 14 can also be configured to perform data compression using a well-known method prior to the transfer operation. In addition to the data compression operation, it can also perform image processing and manipulation (for example, image processing such as leaving only the image portion of the person or object being monitored and deleting unnecessary images, or image processing such as increasing the resolution of the image of the monitored object and decreasing the resolution of the image of other objects). The transfer PC 14 may be connected to the collection and storage server 15 wirelessly or via a wired connection.

[0029] The collection and storage server 15 is a computer that collects and stores in a storage unit (not shown) the video data transferred from the transfer PC 14. The analysis of the collected video data is executed by the analysis server 16, but it is also possible for the collection and storage server 15 to be responsible for part of the analysis process, for example, part of the analysis of the video data.

[0030] The analysis server 16 analyzes the various types of video data stored in the collection and storage server 15, analyzes the movements of people, objects, and machines, and generates analysis data as the analysis results. The generated analysis data is output to the display PC 17 and displayed in a visualized form that can be recognized by an operator. The analysis server 16 can also be configured to compare the analysis data as the result of analyzing the movements of people, objects, and machines with the work plan data and analyze the differences.

[0031] The analysis items in the analysis server 16 may include, but are not limited to, items that a person (e.g., a site supervisor) may discover in the course of daily supervisory duties. For example, a signal A contained in video data acquired and accumulated (learned) over a long period of time in the collection and storage server 15 may be compared with a signal A' contained in newly acquired video data, and the comparison result may be used as the analysis item. If there is a difference between the signals A and A' thus acquired that is equal to or greater than a predetermined value, the analysis server 16 identifies the location where this difference occurred as a "change point" that indicates a significant change in the behavior of a person, object, or machine.

[0032] Identifying such change points makes it possible to detect when significant behaviors of people, objects, and machines occur that are often overlooked by human monitoring, and can serve as an indicator of factors impeding work efficiency in the manufacturing process. Furthermore, with human monitoring, it is necessary to separate data showing correct behavior from data showing abnormal behavior. While correct data is easy to prepare, preparing incorrect data is time-consuming. This system eliminates the need to prepare incorrect data, allowing for easy system implementation in a short period of time. The analysis server 16 links the time and location of such change points and mathematically defines them as "work time at a specific work location."

[0033] Generally, just having people and things in a specific place does not necessarily mean that work is progressing properly. However, by comparing and analyzing the work time data obtained as described above, it is possible to, for example, (a) If the work currently being done deviates from the average work time, (b) When there is a large variation in work time between multiple tasks, (c) When people or things are staying outside of a specific work location These can be detected "mechanically." These are different from the "work time" that humans define or recognize, but they can detect factors that hinder manufacturing lead time. That is, (a) is a hindrance caused by "deviating from the average time," while (b) is a hindrance caused by "large variations" in the work itself. (c) is a hindrance caused by "waiting for work" and "backlog of goods."

[0034] The operation of the human monitoring camera 11 will be explained briefly with reference to FIG. 2. FIG. 2 shows a manufacturing site where multiple machines ((1) to (9)) are arranged, and how a human (worker) H moves along the arrows, captured by the human monitoring camera 11. As an example, the human monitoring camera 11 can store information on the physique of the human H, the characteristics (color, shape, etc.) of the clothes usually worn by the human H, and movement characteristics (average walking speed, stride length). The human monitoring camera 11 can identify the image of the human H included in the video data by comparing this information with the video data. The identified image of the human H is enclosed by a bounding box BB, and the video data can be recorded in a form that includes the bounding box BB.

[0035] In this way, when video data is obtained by the human monitoring camera 11, the video data is analyzed in the analysis server 16, and it is possible to analyze which tasks the person H performed at which time periods (stay time), and which time periods the person H did not perform the task but moved around (movement time), or was in a position outside the capture range, according to the identified change points in the person's behavior. The change points in the person's behavior can be identified according to the time the person arrived in front of a specific machine, the time they left, and other information. The bar display in the lower right of Figure 2 indicates the task times (including start and end times) for tasks (4) to (8). Such bar displays can also be displayed on the display screen of the display PC 17.

[0036] In this embodiment, the worker detection means is configured by the people monitoring camera 11 or an external controller. The people monitoring camera 11 as the worker detection means detects workers in captured images and may include a deep learning network that has previously learned images of workers. This worker detection means detects workers in captured images. From the detected position, it is determined whether the worker is present at a predetermined position, i.e., the time of entry and exit from the work area. In this embodiment, captured images are acquired from multiple directions, and the worker's location (viewpoint direction) in each 2D image is determined. The intersection of these directions is detected as the worker's location in 3D space. This method of detecting the worker's location from multiple images is effective in cases where high depth detection accuracy is desired. Furthermore, using multiple images also has the effect of enabling detection in cases where the worker is hidden in the image due to being shadowed by equipment or other objects.

[0037] The operation of the object monitoring camera 12 and the machine monitoring camera 13 will be explained briefly with reference to Fig. 3. Fig. 2 illustrates an example in which three object monitoring cameras 12 (12A to 12C) and three machine monitoring cameras 13 (13A to 13C) are arranged, but this is merely an example, and the numbers of object monitoring cameras 12 and machine monitoring cameras 13 are not limited to a specific number. Furthermore, the object monitoring camera 12 is not limited to a specific type, but can preferably be a camera that photographs a two-dimensional code 22 attached to a cart 21 on which an object (part, etc.) is placed, as shown schematically in Fig. 3.

[0038] The cart 21 moves between a plurality of machines (11) to (13), as shown in Fig. 3. For example, when the cart 21 completes a process preceding the machine (11) and moves to the front of the machine (11), the two-dimensional code 22 attached to the cart 21 is captured by the object monitoring camera 12A. This allows the arrival of the cart 21 at the machine (11) to be detected.

[0039] When work on machine 11 is finished and cart 21 leaves machine 11 and moves toward the next machine 12, object monitoring camera 12A detects that the two-dimensional code 22 is no longer captured, thereby detecting that cart 21 has left machine 11. When cart 21 arrives in front of machine 12, object monitoring camera 12B captures an image of the two-dimensional code 22 attached to cart 21. This detects that cart 21 has arrived at machine 12. By repeating this operation, the movement of cart 21, i.e., the object (parts, etc.), can be detected based on the video data from object monitoring camera 12.

[0040] Change points in the behavior of an object can be identified by the start and end times of a certain machine (see Figure 4). The travel time (waiting time) from a higher-level machine (process) to a lower-level machine (process) can be calculated by the difference between the departure time from the higher-level machine and the arrival time at the lower-level machine. Change points in machine behavior can also be identified by the start and end times of a certain machine's specific operation.

[0041] Furthermore, when parts are placed on the cart 21 from the serving shelf, the intermediate part products placed on the cart 21 are assigned a correspondence relationship with the cart number, part number, and product serial number linked to the part number. The correspondence relationship is also assigned as information to the two-dimensional code 22, and by reading the two-dimensional code 22, this correspondence information is automatically collected. This series of automatic collection links the cart 21, part, worker, shop, and change time. This linking is also performed between other shops, other products, and other workers. As a result, the locations of parts, workers, facilities, equipment, and jigs at a specific time are recognized, and the vector data is stored in the storage means. Furthermore, the detection of the time of a status change does not necessarily have to be based on images captured by a TV camera, but can also be combined with information from other sensors, already installed sensors, etc.

[0042] The operation of the monitoring system 1 according to the embodiment will be described with reference to the flowchart in Figure 5. In this system, the people monitoring camera 11, the object monitoring camera 12, and the machine monitoring camera 13 each operate independently to monitor the operations of people, objects, and machines. The obtained video data is transferred to and read by the transfer PC 14, which then performs a predetermined data compression operation on the video data and transfers the video data to the collection and storage server 15. The collection and storage server 15 receives and stores the transferred data. The data stored in the collection and storage server 15 is sent to the analysis server 16 as appropriate and is used for analysis.

[0043] Based on the stored data, the analysis server 16 identifies the change points of people, things, and machines as described above, identifies the time when the change points occurred, and further identifies the location where the change points occurred, and stores this information in a storage unit (not shown). By identifying multiple change points, it is possible to identify the start time and end time of significant actions of people and each machine of things.

[0044] By identifying change points and the time and place where the change points occurred, the status of people and things in each process can be determined, which not only determines the time required for each process but also determines which of multiple processes has room for shortening the lead time (room for margin). In other words, the analysis server 16 can function as a room for margin determination means for determining room for margin in each process based on the identified change points and data at the time when significant actions occurred. Room for margin can be visually displayed on the display PC 17.

[0045] FIG. 6 is an example of a display screen when the analysis results in the analysis server 16 are displayed on the display of the display PC 17. In the display example of FIG. 6, the horizontal axis is the time axis, and it is a Gantt chart in which the timeline of the manufacturing plan and the timeline of each process (processes 1 to 3) in the actual work (performance) are compared by bar display for each different production number. This display visualizes the difference between the manufacturing plan and the actual work on each work day. This screen display visualizes which production number is in which process, and the progress of each sub-process within a large process that takes place over several days can be grasped.

[0046] As shown here, the monitoring system of this embodiment is highly effective in monitoring the progress of a manufacturing plan. A delay in the progress of a manufacturing plan leads to increased manufacturing costs due to increased labor costs, equipment depreciation, and other factors. This is why progress management is so effective. Furthermore, when a significant delay occurs, appropriate manufacturing output can be obtained by reassessing the manufacturing plan (manufacturing schedule). This is why rescheduling of the manufacturing plan is necessary. In this case, the magnitude of the delay can be preset, for example, a 20% or 30% increase from the initial lead time. Alternatively, the takt time of a specific process that is causing a bottleneck can be preset, for example, a 20% or 30% increase. In other words, the monitoring system of this embodiment plays an important role in determining when rescheduling is necessary. In other words, using the monitoring system of this embodiment in combination with a scheduler can produce even greater benefits. In this case, a certain degree of effectiveness can be expected whether the scheduler being tried out is an automatic stacking scheduler or a scheduler equipped with some kind of optimization algorithm.

[0047] FIG. 7 is another example of a display screen when the analysis results from the analysis server 16 are displayed on the display of a display PC. In the display example at the top of FIG. 7, the horizontal axis represents elapsed time, and the breakdown of the work time for each process (processes 1 to 3) for each different production number is displayed as the length of the bar. This display visualizes the variation in the total work time for each production number, as well as the variation in the time required for each process. This makes it easy to identify production numbers that require long work times and to investigate the causes.

[0048] In the example display at the bottom of Fig. 7, the horizontal axis represents elapsed time, but the travel time between different processes for each of multiple carts is displayed as the length of the bar. This display makes it possible to visualize which carts and which processes have long travel times, making it easy to identify the causes.

[0049] Figure 8 is a graph of the operating time and non-operating time for each process (processes 1 to 3). When focusing on a single process, the higher the ratio of operating time to the total (operating rate), the higher the work efficiency. However, if a process has a high operating rate while an adjacent process has a low operating rate, the work efficiency of the entire process cannot be improved. When there is a difference in operating rate between adjacent processes in this way, it can be determined that a bottleneck has occurred in the work. When a bottleneck occurs, work efficiency can be improved by changing the manufacturing plan, including personnel allocation.

[0050] In the embodiment described above, for security reasons, there are cases where the user wants to avoid uploading on-site information (information on the processing of image data after acquisition in this embodiment, time detection of changes in the status of people, objects, and equipment, and the results of subsequent data processing) to the cloud. In this case, the change time, product information, and other information can be encoded and decoded in advance using a password set by the user. This allows this information to be treated as text information whose content has meaning to different people within the site, and as information that has no meaning without a password on the cloud.

[0051] Furthermore, by identifying data on the cloud using a hash function before and after use, it is possible to ensure that the data on the cloud has not been tampered with, thereby ensuring the security of user data. This method may also utilize blockchain technology, which adds a previous hash value to stored data to generate the next hash value, thereby making it virtually impossible to tamper with the data, thereby further improving security.

[0052] Furthermore, in systems that use images from TV cameras, workers may be resistant to being photographed or may be concerned about their personal information being leaked. To address this issue, a controller or other device can be provided with a means for converting captured images into information that cannot be interpreted by the human eye, such as by applying image transformations such as Fourier transforms, at an early stage. By using such converted images in subsequent learning and other processes, similar image transformations can be performed in subsequent processes, such as worker detection.

[0053] By acquiring various time data, including the detected work completion time, for each product, the lead time from start to finish of the product can be analyzed as the work time for each work. Also, for example, at the end of a day's work, a report can be created that compiles the progress of work, the placement positions of parts, etc.

[0054] A system and method for utilizing various time data such as take-off and arrival times and start and completion times detected in this way will be explained below. With this system, it is possible to describe the progress time of each task by focusing on a specific product. When the variance in the progress time is calculated for a specific product or a specific shop, if the variance is large due to the worker or other disturbances, it can be determined that the task is not mature. In such a case, for example, · Change the worker for a specific task to a member with higher work skills Analyze the causes of variation and prepare jigs to separate the work Countermeasures such as the following can be taken.

[0055] In the above case, by monitoring the variability in start-to-completion times, it is possible to know in real time whether the quality of work is deteriorating.

[0056] Furthermore, by preparing an environment on the cloud that includes the monitoring system, production planning system, and display system shown in Figures 6 to 8 of this embodiment, the following new value can be created. For example, when an algorithm is developed that can detect human change points with greater accuracy and higher time resolution, the value of the entire system can be improved by simply changing this algorithm. For example, because a process can be broken down into smaller parts when it is changed, changing the combination of these parts can further improve the accuracy of lead time reduction through process redesign. In this way, having a monitoring system, production planning system, or display system allows new technologies and programs to be easily and quickly applied to the workplace. In other words, the present invention has the aspect of providing a "platform" for improving the efficiency of the production site.

[0057] Furthermore, by counting the usage time of each technology and program applied on-site, it is possible to measure the total usage time of those technologies. The system provider's compensation can be automatically calculated based on this usage time. This is the mechanization of administrative processes, but even in manufacturing sites, the mechanization and efficiency of such administrative processes is important in terms of cost reduction. Furthermore, by having this type of usage time measurement system and paying compensation according to usage time, one-time cash payments can be reduced compared to introducing a system with a lump-sum payment, which is expected to have the effect of improving cash flow.

[0058] Furthermore, this monitoring system can detect and report deviations from the average work time and expected work completion time. In a remote monitoring system using a simple surveillance camera, the monitor must continuously monitor the images being captured, whereas the system can be expected to report only when deviations from the expected time occur.

[0059] [effect] As described above, according to the system of this embodiment, the movements of people, objects, and machines are monitored by cameras 11-13. Significant changes in the movements of people, objects, and machines are detected based on change points in the video data. The analysis server 16 then analyzes the results and visualizes them for display. Therefore, the movements of people, objects, and machines can be monitored visually by an operator, eliminating the need for time-consuming input and setting of conditions. This allows for accurate and low-cost monitoring of the manufacturing process, improving work efficiency. In other words, matters that are currently judged intuitively by a supervisor can be handled with the same level of accuracy by capturing images using cameras 11-13. As a result, this system allows for efficient monitoring of the manufacturing site without the need for a dedicated system engineer. Furthermore, visualization of the manufacturing site allows for visualization, which in turn enables one-stop solutions to business issues.

[0060] The cameras 11-13 used can be installed at different locations, with different types or numbers being increased as needed, making it easy to adapt to changes in the manufacturing site. Specifically, this system can record the status or results of work sites where it is difficult to record the status or results, such as human assembly work, human visual inspection, human transport of goods, and human machine operation. As a result, the status of a human work site can be recorded in the same way as a factory site composed of automated machines that has a mechanism for recording data indicating the status.

[0061] [others] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. [Explanation of symbols]

[0062] 1...surveillance system, 11...human surveillance camera, 12...object surveillance camera, 13...device surveillance camera, 14...transfer computer (PC), 15...collection and storage server, 16...analysis server, 17...display computer (PC), 21...cart, 22...two-dimensional code.

Claims

1. In a monitoring system that monitors people, things, and equipment in the manufacturing process, a camera that photographs a plurality of work processes in a plurality of work locations and monitors the people, objects, and devices; an analysis unit that analyzes the video data obtained by the camera; a display unit that displays the analysis results by the analysis unit; Equipped with The analysis unit Detecting a change point in the video data; Analyzing the states of the person, object, and device based on the change points; comparing signals contained in newly acquired video data with signals contained in video data that have been machine-learned to detect the change points, and detecting and linking the positions and times of the change points to define work time at a specific work location, thereby monitoring work errors in the manufacturing process; the analysis unit detects the change points in the movement of the person according to the time when the person arrives in front of a predetermined machine and the time when the person leaves the machine, the analysis unit detects the change point in the operation of the item according to a start time and a stop time of a predetermined machine for the item; the analysis unit detects the change points in the operation of the device according to the time when the device starts and the time when the device finishes a specific operation; The analysis unit identifies the change points and the times and places at which the change points occurred, thereby identifying the states of the people and the things in each of the work processes, identifying the time required for each of the work processes, and further determining whether there is room for shortening the lead time in any of the work processes. A monitoring system characterized by:

2. The analysis unit 2. The monitoring system according to claim 1, further comprising an inspection object detecting means for detecting an inspection object that is a candidate for the change point.

3. The analysis unit The monitoring system according to claim 1, further comprising a means for sending the machine-learned video data parameters to a manufacturing process base and performing a weighted average with the video data parameters learned at the base.

4. The monitoring system according to claim 1, characterized in that a plurality of cameras are installed to monitor the people, objects, and devices, thereby detecting the positions of the people, objects, and devices in three-dimensional space.

5. In a monitoring system that monitors people, things, and equipment in the manufacturing process, sensors that monitor the people, objects, and devices in a plurality of work processes at a plurality of work locations; an analysis unit that analyzes sensing data obtained by the sensor; a display unit that displays the analysis results by the analysis unit; Equipped with The analysis unit Detecting a change point in the sensing data; Analyzing the states of the person, object, and device based on the change points; The signal included in the newly acquired sensing data is compared with the signal included in the machine-learned sensing data to detect the change point, and the time and location of the change point are detected and linked to define the work time at a specific work location, thereby monitoring the manufacturing process. the analysis unit detects the change points in the movement of the person according to the time when the person arrives in front of a predetermined machine and the time when the person leaves the machine, the analysis unit detects the change point in the operation of the item according to a start time and a stop time of a predetermined machine for the item; the analysis unit detects the change points in the operation of the device according to the time when the device starts and the time when the device finishes a specific operation; The analysis unit identifies the change points and the times and places at which the change points occurred, thereby identifying the states of the people and the things in each of the work processes, identifying the time required for each of the work processes, and further determining whether there is room for shortening the lead time in any of the work processes. A monitoring system characterized by:

6. 6. The monitoring system according to claim 5, wherein the display unit includes a scheduler.

Images