Management System

Abstract

Conventional management systems have limitations in the types of data they can process using images, making them impractical. [Solution] A management system is provided comprising: a shooting device that photographs an environment including one or more objects to be managed and acquires video; a computer connected to the shooting device and having a display unit that displays the video, and equipped with an image processing unit that has a function to process the video, wherein the image processing unit can set one or more detection frames in the video, can specify a specific color for each detection frame, and for each detection frame, acquires the date and time when the specified specific color was displayed in the detection frame, and the date and time when the specified specific color was no longer displayed in the detection frame, and counts the number of objects to be managed that have the target color corresponding to the specified color.

Description

Technical Field

[0001] The present disclosure relates to a management system for managing a managed object.

Background Art

[0002] Conventionally, as a management system for managing operation information, location information, etc. of a managed object, a system that detects the managed object with a photoelectric sensor and records (logs) the detection date and time, the number of detections, etc. of the managed object using an amplifier and a PLC (Programmable Logic Controller) may be used. However, generally, photoelectric sensors and PLCs are expensive, and the introduction cost is high. Therefore, an inexpensive and simple management system is required. In addition, when using a photoelectric sensor, the reflection intensity of light varies depending on the type of the managed object, and the detectable range also varies, so there are limitations on the managed object.

[0003] Further, Patent Document 1 discloses a monitoring system for monitoring the situation of pile driving construction, which includes a photographing device that photographs the screen of a display installed at the location where the pile driving construction is being carried out, a recording device that records the state of the pile driving construction, an analysis device that acquires the image of the screen photographed by the photographing device and the video of the state recorded by the recording device, and a monitoring display. The analysis device performs a recognition process of analyzing the image of the screen and recognizing, for each management item related to the situation of the pile driving construction, the numerical value of the information displayed on the screen during the pile driving construction, a storage process of storing the recognized numerical value in a storage device in association with the management item, and a display process of displaying both the visualization information about the management item based on the numerical value and the video of the state on the monitoring display. Such a monitoring system requires preparing two types of devices, a photographing device and a recording device, and developing a dedicated system for integrating and displaying both of them.

[0004] Patent Document 2 discloses a system for managing operations using a camera and a computer having a display unit and an image processing unit. Specifically, it describes an automatic dissolution detection system that photographs the reaction product with a webcam and monitors the color tone of the reaction product using commercially available image processing visual programming tools to determine the timing of reaction completion based on the RGB values. Since such a system is a management system that uses only a camera or other photographic device and a computer, it is an inexpensive and simple management system. However, if the color change of the reaction product is small, the ability to make a judgment will depend on the object being measured, and a dedicated image sensor that can read color changes more accurately will be required. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2024-080796 [Patent Document 2] Patent No. 7369495 [Overview of the project] [Problems that the invention aims to solve]

[0006] A management system consisting of an imaging device and a computer with an image processing unit is desired as an inexpensive and simple management system. However, conventional management systems have limitations in the types of data that can be processed, making them impractical.

[0007] This disclosure was made in view of the above-mentioned problems, and aims to provide a management system that can easily and inexpensively obtain the detection date and time and the number of detections of a managed object, without any limitations on the objects being managed. [Means for solving the problem]

[0008] This disclosure provides a management system for managing one or more objects to be managed, comprising: a shooting device that photographs an environment including the one or more objects to be managed and acquires video; and a computer connected to the shooting device, having a display unit for displaying the video and equipped with an image processing unit that has a function for processing the video, wherein the image processing unit can set one or more detection frames in the video, can specify a specific color for each detection frame, and for each detection frame, acquires the date and time when the specified specific color was displayed in the detection frame, and the date and time when the specified specific color was no longer displayed in the detection frame, and counts the number of objects to be managed that have been assigned a target color corresponding to the specified color. [Effects of the Invention]

[0009] The management system described in this disclosure has the effect of allowing for the easy and inexpensive acquisition of the detection date and time and the number of detections of managed objects, without any limitations on the objects being managed. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram illustrating the management system described in this disclosure. [Figure 2] This diagram illustrates the computer processing described in this disclosure. [Figure 3] This diagram illustrates the controlled items and detection frameworks in this disclosure. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described below with reference to drawings and other figures. However, this disclosure can be implemented in many different ways and should not be interpreted as being limited to the embodiments described below. In addition, in order to make the explanation clearer, the drawings may schematically represent the width, thickness, shape, etc. of each part compared to the actual form, but these are merely examples and should not limit the interpretation of this disclosure. Furthermore, in this specification and each figure, elements similar to those described above with respect to previously shown figures will be denoted by the same reference numerals, and detailed explanations may be omitted as appropriate.

[0012] The management system for this disclosure is described below.

[0013] This disclosure provides a management system for managing one or more objects to be managed, comprising: a shooting device that photographs an environment including one or more objects to be managed and acquires video; a computer connected to the shooting device and having a display unit that displays the video, and equipped with an image processing unit that has a function to process the video, wherein the image processing unit can set one or more detection frames in the video, can specify a specific color for each detection frame, and for each detection frame, acquires the date and time when the specified specific color entered the detection frame and the date and time when the specified specific color left the detection frame, and counts the number of objects to be managed that have the target color corresponding to the specified color.

[0014] Figure 1 is a schematic diagram illustrating the management system of the present disclosure. As shown in Figure 1, the management system 10 of the present disclosure is a management system that manages one or more objects to be managed (object A, object B, object C), and comprises a camera 1 and a computer 2.

[0015] As shown in Figure 1, the imaging device 1 captures an environment including one or more objects to be managed (object A, object B, object C), acquires video V, and outputs video V to the computer 2. The computer 2 is connected to the imaging device 1 for communication purposes, has a display unit 21 that displays video V, and is equipped with an image processing unit 22 that has the function of processing video V. In Figure 1, objects A, B, and C have color indicator materials of target color a', b', and c' attached to them, respectively. Objects A, B, and C shown in Figure 1 are mobile objects that move in the vertical direction. The imaging device 1 is connected to the computer 2 by a USB cable.

[0016] The image processing unit 22 processes the output video V. Specifically, the image processing unit 22 sets one or more detection frames (detection frame X, detection frame Y, and detection frame Z) at predetermined positions and within predetermined ranges in the video V captured by the imaging device 1. Furthermore, a specific color is specified for each detection frame using RGB values. In Figure 1, specific color a is specified for detection frame X, specific color b is specified for detection frame Y, and specific color c is specified for detection frame Z.

[0017] The image processing unit 22 then obtains, for each detection frame, the date and time when a specified specific color was displayed within the detection frame, and the date and time when the specified specific color ceased to be displayed within the detection frame. For example, if the area in which the specified color a is displayed exceeds a predetermined threshold relative to the area of ​​detection frame X, it is determined that the specified color a is displayed in detection frame X. On the other hand, if the ratio of the area in which the specified color a is displayed to the area of ​​detection frame X falls below a predetermined threshold, it is determined that the specified color a ceased to be displayed within detection frame X. This process is repeated, and the number of times the specified color a is displayed within detection frame X is counted as the number of detected managed objects A that have been assigned the target color a' corresponding to the specified color a. Similarly, for detection frame Y, the date and time when a specified specific color b is displayed within detection frame Y, and the date and time when the specified specific color b ceased to be displayed within detection frame Y are measured in the same manner, and the number of detected managed objects B that have been assigned the target color b' corresponding to the specified color b is counted. Similarly, for detection frame Z, the date and time when the specified color c is displayed within detection frame Z, and the date and time when the specified color c is no longer displayed within detection frame Z are measured, and the number of detected target objects C that have the target color c' corresponding to the specified color c is counted.

[0018] As shown in Figure 1, it is preferable that the computer 2 in this disclosure further includes a conversion unit 23 that converts the number of detections into data along with date and time information and outputs it, and a recording unit 24 that records the output data.

[0019] Conventional photoelectric sensor-based management systems have limitations on what can be targeted because the light reflection intensity and detectable range differ depending on the type of object being managed. On the other hand, the management system of this disclosure acquires the date and time when a specific color is displayed within the detection frame and the date and time when it is no longer displayed, and counts the number of detected objects that have the target color corresponding to the specific color. Therefore, the entry and exit of objects into the detection frame is determined solely by color elements (for example, the three elements of RGB). As a result, objects only need to have the target color, and various types of objects, people, etc., can be targeted.

[0020] For example, by attaching a color display material of the target color to the operating part of a work improvement jig, tools, and the movable parts of devices, etc., it can be used as a management target, and it becomes possible to manage the operating status of the operating part of the work improvement jig, the entry and exit status of tools in the tool storage area and tool rack, and the movable status of the movable parts of the device, etc.

[0021] On the other hand, when the management target itself has a target color that can be distinguished from the background, etc., there is no need to attach a color display material. For example, by using a colored magnet as the management target and photographing the whiteboard with a photographing device, it is also possible to manage the entry and exit status of the colored magnet on the whiteboard. Similarly, by using a person wearing a colored helmet as the management target and photographing the corridor with a photographing device, it is also possible to manage the number of people passing through the corridor, for example, by business type or job title.

[0022] According to the management system of the present disclosure, since there is no limit to the number of detection frames, it is possible to measure the detection time and the number of detections for all management targets reflected in the photographing range of the camera. For example, when it is desired to manage a plurality of management targets at once, there is no need to increase the sensors or photographing devices, and by providing a plurality of detection frames, it becomes possible to measure the detection time and the number of detections of each management target.

[0023] In addition, the accuracy of detecting the management target by the conventional management system may be affected by the processing capabilities of the web camera, PC, and image processing software. On the other hand, in the management system of the present disclosure, it is possible to adjust and set the specific color specified for each detection frame according to the performance of the photographing device. Therefore, it becomes possible to accurately detect the management target.

[0024] Furthermore, since the management system of the present disclosure can perform work management only with a terminal having a photographing device and an image processing unit, there is no need to introduce generally expensive optoelectronic sensors, PLCs, etc., and it can be constructed at a low cost and easily.

[0025] As described above, the management system described in this disclosure can easily and inexpensively obtain the detection date and time and detection count of managed items without any limitations on the items to be managed, thereby promoting the digital transformation of management.

[0026] 1. Imaging device The management system in this disclosure includes a camera that captures an environment containing one or more objects to be managed. The camera is connected to a computer, enabling it to send and receive data to and from the computer. The camera can be configured, for example, with a webcam, a digital video camera, or other device with video recording capabilities. A webcam is preferred because it can transmit video to the computer in real time. If the computer, which will be described later, has a camera function, the camera may be configured to be integrated with the computer in a way that utilizes this function. The camera is preferably fixed to the ceiling, wall, floor, or the like.

[0027] 2. Computer The management system in this disclosure has a computer connected to a camera. As shown in Figure 1, the computer 2 has a display unit 21 and an image processing unit 22. The computer 2 may also have a conversion unit 23 and a recording unit 24, which will be described later. The computer also typically has a keyboard (input means), a mouse (selection means), a CPU, ROM, RAM, etc. In this disclosure, the computer is connected to the camera in a communicative manner and receives the video to be displayed. It is preferable that this video is acquired in real time. The computer may also be a device equipped with a shooting function that can function as a camera.

[0028] The image processing unit can, for example, use an image processing library that can run on a computer, such as the publicly available image processing library called OpenCV.

[0029] The image processing unit can set one or more detection frames within the video, and can specify a particular color for each detection frame. For each detection frame, it obtains the date and time when the specified color was displayed within the frame, and the date and time when the specified color was no longer displayed within the frame, and counts the number of detected objects that have the target color corresponding to the specified color.

[0030] The position and size of the detection frame within the video are not particularly limited and are set appropriately to a position and size suitable for detecting the target object. The number of detection frames set within the video is not particularly limited and may be one or two or more. Each detection frame is intended to detect one target object.

[0031] When setting two or more detection frames, they may be set so that they do not overlap, or they may be set to overlap. When setting two or more detection frames to overlap, the entire area of ​​one detection frame may overlap with part or all of another detection frame, or a part of the area of ​​one detection frame may overlap with part or all of another detection frame.

[0032] Figure 3 is a diagram illustrating the objects to be managed and the detection frames in this disclosure. For example, the two detection frames X and Y shown in Figure 3(a2) are set so as not to overlap with each other. On the other hand, the two detection frames X and Y shown in Figure 3(b2) are set so that parts of each other overlap. Detection frame X is the frame for detecting object A shown in Figures 3(a1) and (b1), and detection frame Y is the frame for detecting object B shown in Figures 3(a1) and (b1). Furthermore, the two detection frames W1 and W2 shown in Figure 3(c2) are set so as to completely overlap with each other. Both detection frames W1 and W2 are frames for detecting object D shown in Figure 3(c1).

[0033] In the image processing unit, for example, a specific color is specified within a range of RGB values ​​for each detection frame. For example, as shown in Figures 3(a2) and (b2), specific color a is specified for detection frame X, and specific color b is specified for detection frame Y. Here, specific color a is specified within a predetermined range of RGB values ​​so as to include the RGB values ​​of the display color of target color a' attached to the object being managed A. Similarly, specific color b is specified within a predetermined range of RGB values ​​so as to include the RGB values ​​of the display color of target color b' attached to the object being detected B. As an example of how to determine the specific range of RGB values, first, the object being managed (colored paper, etc.) is displayed in the app by being captured by the camera. Next, with 100% being the case when all pixels of the object being managed displayed within the detection frame match the RGB values ​​within the specified range in the app, the minimum and maximum values ​​of each RGB element are manually manipulated and set so that the threshold for pixels detecting the object being managed in the app is, for example, around 90-95%.

[0034] When setting up two or more detection frames, they may be assigned the same specific color, or they may be assigned different specific colors, but it is preferable that they be assigned different specific colors. For example, when setting up detection frames as shown in Figures 3(a2) and 3(b2), it is preferable that the specific color a of detection frame X and the specific color b of detection frame Y are different. In this case, the target color a' of the target object A detected by detection frame X and the target color b' of the target object B detected by detection frame Y are usually different. As a result, for example, even if the target object B that should be detected by detection frame Y is captured within detection frame X, it will not be determined that the specific color of detection frame X (i.e., specific color a) is displayed, thus suppressing false detections of the target object and enabling accurate management.

[0035] For example, even when setting two detection frames W1 and W2 overlapping as shown in Figure 3(c2), it is preferable that the specific color d in detection frame W1 and the specific color e in detection frame W2 are different. In this case, it is preferable that the objects to be detected by detection frames W1 and W2 are the same and are objects to be detected (object D) that have two target colors (target colors d', e'). By setting detection frames W1 and W2 overlapping in this way and preparing objects to be detected with two target colors, the detection accuracy of object D can be improved by obtaining the date and time when the specific color d is displayed in detection frame W1 and the specific color e is displayed in detection frame W2, and the date and time when the specific color d is no longer displayed in detection frame W1 and the specific color e is no longer displayed in detection frame W2.

[0036] In the image processing unit, for example, if the number of pixels detecting the target color to be managed exceeds a predetermined threshold, it is determined that the specific color is displayed in the detection frame (ON). On the other hand, if the number of pixels detecting the target color to be managed falls below a predetermined threshold, it is determined that the specific color is no longer displayed in the detection frame (OFF).

[0037] The above threshold can be appropriately changed depending on the performance of the imaging equipment and computer, and may be, for example, 10% or more, 50% or more, or 90% or more. This is because detection accuracy is improved by setting the threshold within the above range. On the other hand, the above threshold may be, for example, 100% or less, or 95% or less.

[0038] The image processing unit repeats the above process and counts the number of times the specific color is displayed (ON) within the detection frame as the number of detected objects. Alternatively, the detection time of the object (the time the object is within the detection frame) may be calculated from the date and time when the specific color is displayed within the detection frame and the date and time when the specific color is no longer displayed within the detection frame. Similarly, the non-detection time of the object (the time the object is not within the detection frame) may be calculated from the date and time when the specific color is no longer displayed within the detection frame and the date and time when the specific color is next displayed within the detection frame.

[0039] The computer preferably has a conversion unit that converts the number of detections into data along with date and time information and outputs it. For example, as shown in Figure 2(a), the number of detected objects under management at a predetermined date and time can be converted into data. It is preferable that the data is updated in real time. Also, as shown in Figure 2(a), if necessary, video from the camera may be recorded and the video information may be compared with the number of detections.

[0040] The computer preferably has a recording unit that processes and records the output data. For example, as shown in Figure 2(b), it is preferable to process the data to record the number of detected objects under management at a predetermined date and time in a graph or the like.

[0041] 3.Target to be managed The object to be controlled is not particularly limited as long as it has the target color. The object to be controlled may have the target color by attaching a color-marking material colored with the target color. Examples of color-marking materials include colored paper, colored resin boards, colored tape, etc. On the other hand, the object to be controlled itself may have the target color in part or all of it.

[0042] A single object under management may have one or more target colors, or it may have two or more target colors, as shown in Figure 3(c1).

[0043] The target color is preferably different from the background color of the object being managed (for example, the color of the floor, wall, desk, etc.). The target color may be chromatic or achromatic, but chromatic is preferred because it improves detection accuracy.

[0044] When managing two or more objects, it is preferable that the target colors of each object be distinguishable from each other. For example, when managing two objects, it is preferable to use two colors that are opposite to each other. For example, when managing three objects, it is preferable that the target colors be red, green, and blue. For example, if the colors are similar, such as red and pink, false detection may occur depending on the shooting environment and the performance of the shooting equipment.

[0045] Objects subject to management include, for example, objects that move, such as things and people. Examples include tools, the moving parts of work improvement jigs, and the movable parts of devices. Examples of tools include wrenches, screwdrivers, scissors, hammers, pliers, nippers, and other similar tools. In this specification, a work improvement jig is a tool or device that can improve work efficiency by moving its moving parts mainly by human power, unlike devices that are generally powered by means other than human power. Examples of work improvement jigs with moving parts include cardboard box unpacking and assembly jigs. Examples of devices with movable parts include printing devices, packaging devices, and conveying devices.

[0046] For example, by designating tools as the target of management and photographing tool storage areas and tool shelves with cameras, it is possible to understand the status of tool entry and exit, as well as their organization and tidiness. Furthermore, by designating the moving parts of work improvement jigs as the target of management and photographing the range of motion of these parts, it is possible to understand when and how many times workers moved the moving parts.

[0047] For example, colored magnets can be used as the objects to be managed. By using colored magnets as the objects to be managed and photographing the whiteboard with a camera, it is possible to record the movement of colored magnets onto and off the whiteboard. Such a management system can be used for attendance management in a company. Another example of objects to be managed is people wearing colored helmets. By using people wearing colored helmets as the objects to be managed and photographing the hallway with a camera, it is possible to record the number of people passing through the hallway, for example, by assigned job or position.

[0048] The items to be managed may include non-movable items such as tool storage areas and tool shelves. In this case, the date and time when a specific color is displayed within the detection frame corresponds to, for example, the date and time when the tools were taken out, and the date and time when the specific color no longer appears within the detection frame corresponds to, for example, the date and time when the tools were returned. Therefore, the status of tool entry and exit, and the state of organization and tidiness can be managed based on the number of detected items to be managed.

[0049] The present invention is not limited to the embodiments described above. The embodiments described above are illustrative, and any configuration that is substantially identical to the technical idea described in the claims of the present invention and achieves similar effects is included within the technical scope of the present invention.

[0050] Thus, the present disclosure provides, for example, the following inventions.

[0051] [1] A management system that manages one or more objects to be managed, A camera that captures images of an environment including one or more of the aforementioned objects to be managed, and acquires video footage. The system includes a computer connected to the aforementioned imaging device, having a display unit for displaying the video, and equipped with an image processing unit that has a function for processing the video, The aforementioned image processing unit, One or more detection frames can be set within the aforementioned video, and a specific color can be specified for each of the detection frames. A management system that, for each detection frame, obtains the date and time when the specified color was displayed within the detection frame, and the date and time when the specified color ceased to be displayed within the detection frame, and counts the number of detected objects to be managed that have the target color corresponding to the specified color. [2] The management system according to [1], wherein the target color is a different color from the background color of the subject to management. [3] The management system described in [1] or [2], wherein the object to be managed is to which the color indicator material of the target color is attached. [4] The aforementioned computer further, A conversion unit that converts the number of detections into data along with date and time information and outputs it, A management system according to any one of [1] to [3], comprising a recording unit for recording the output data. [Explanation of Symbols]

[0052] 1. Photographing equipment 2… Computer 21...Display section 22… Image Processing Unit 23… Conversion section 24… Records Department 10… Management System

Claims

1. A management system that manages one or more objects to be managed, A camera that captures images of an environment including one or more of the aforementioned targets and acquires video footage, The system includes a computer connected to the aforementioned imaging device, having a display unit for displaying the video, and equipped with an image processing unit that has a function for processing the video, The aforementioned image processing unit, One or more detection frames can be set within the aforementioned video, and a specific color can be specified for each of the detection frames. A management system that, for each detection frame, obtains the date and time when the specified color was displayed within the detection frame, and the date and time when the specified color ceased to be displayed within the detection frame, and counts the number of detected objects to be managed that have the target color corresponding to the specified color.

2. The management system according to claim 1, wherein the target color is a different color from the background color of the item to be managed.

3. The management system according to claim 1, wherein the object to be managed is fitted with a color indicator of the target color.

4. The aforementioned computer further, A conversion unit that converts the number of detections into data along with date and time information and outputs it, The management system according to claim 1, further comprising a recording unit for recording the output data.

Images