Control system and control method

Abstract

Comply with factory rules and appropriately avoid danger. [Solution] A control system for avoiding danger to a monitored object using a computer having a processor and memory, in which the processor reads rule violation setting information that has been stored in advance in memory and that defines rules according to the monitoring location of the factory, and determines that the monitored object is violating the rules if the rules defined in the rule violation setting information are not met in the area in front of the stop line where the monitored object should stop and the area beyond the stop line, which are included in an image of the monitoring location of the factory captured by a monitoring camera.

Description

[Technical Field]

[0001] The present invention relates to a control system and a control method. [Background technology]

[0002] Recently, there has been an increasing need for safety and health in factories. This need is not only for preventing accidents and improving productivity on-site, but also for changing employee awareness and providing educational value, and is on the rise. In light of this background, there are various technologies for avoiding dangers that may arise from vehicles entering intersections, etc. For example, Patent Document 1 describes a technology that "includes a camera-driven image processing device 62 that detects the position and speed of vehicles approaching a stop line, which is a stopping position, and a red light violation monitoring unit 65a that predicts vehicles that will pass without stopping at the stop line by determining, based on the position and speed detected by the camera-driven image processing device 62, whether the vehicle will have difficulty stopping at the stop line." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-207177 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, by monitoring vehicles engaging in dangerous behavior such as ignoring traffic signals, nearby pedestrians are alerted to avoid danger. However, there is no mention of whether forklifts and workers moving within a factory are breaking rules such as stopping at necessary locations within the factory, or whether they are violating rules such as stopping at locations within the factory where special caution is required. In other words, there is no consideration given to applying rules appropriate for the factory and appropriately avoiding danger within the factory, taking into account the work environment within the factory, such as the configuration of the roads within the factory and the forklift movement speed that should be observed depending on the location within the factory.

[0005] An object of the present invention is to provide a technique that enables compliance with rules within a factory and appropriate avoidance of danger. [Means for solving the problem]

[0006] The control system of the present invention is a control system for avoiding danger to a monitored object using a computer having a processor and memory, wherein the processor reads rule violation setting information that has been stored in advance in the memory and that defines rules according to the monitoring location of a factory, and if the rules defined in the rule violation setting information are not met in the area in front of the stop line where the monitored object should stop and the area beyond the stop line, which are included in an image of the monitoring location of the factory captured by a monitoring camera, the control system determines that the monitored object is violating the rules. [Effects of the Invention]

[0007] According to the present invention, it is possible to comply with rules within a factory and appropriately avoid danger. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a control system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of a schematic configuration of a computer. [Figure 3] FIG. 1 is a diagram illustrating an example of an environment in which a control system is applied in a factory. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of a recording PC. [Figure 5] FIG. 2 is a block diagram showing the functional configuration of a PC for risky behavior analysis. [Figure 6] 10 is a flowchart showing the processing procedure of processing performed in the present system. [Figure 7] FIG. 10 is a diagram illustrating an image of the processes in S601 and S602. [Figure 8]FIG. 10 is a diagram illustrating an example of a monitoring target type table. [Figure 9A] FIG. 10 is a diagram for explaining a method for determining whether or not a rule violation has occurred. [Figure 9B] FIG. 10 is a diagram showing an example of information (rule violation setting information) relating to a predetermined rule violation. [Figure 10] FIG. 10 is a diagram for explaining a method for determining whether or not a rule violation has occurred (when a stop line has been crossed). [Figure 11] FIG. 10 is a diagram illustrating an example of calibration information. [Figure 12] FIG. 4 is a diagram illustrating an example of stop line information. [Figure 13] FIG. 10 is a diagram illustrating an example of analysis result information. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The examples are illustrative for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0010] Examples of various types of information may be described using expressions such as "table," "list," and "queue," but the various types of information may also be expressed using data structures other than these. For example, various types of information such as "XX table," "XX list," and "XX queue" may also be expressed as "XX information." When describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0011] When there are multiple components with the same or similar functions, they may be described using the same reference numeral with different subscripts. When there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0012] In the embodiments, processing performed by executing a program may be described. Here, a computer executes the program using a processor (e.g., a CPU or a GPU) and performs processing defined by the program using storage resources (e.g., a memory) and interface devices (e.g., a communication port). Therefore, the entity performing the processing by executing the program may be the processor. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be any computing unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit may be, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

[0013] A program may be installed on a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in an embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0014] 1 is a diagram showing an example of the configuration of a control system in this embodiment. The control system 1000 in this embodiment is a system that enables forklifts traveling within a factory and workers moving around to comply with rules within the factory and appropriately avoid danger. In the following, forklifts and workers are used as examples of objects moving within a factory, but the present invention is not limited to these and can be applied to various moving objects such as autonomously moving robots and carts.

[0015] As shown in FIG. 1, control system 1000 includes a monitoring camera 100 for capturing images of a predetermined area within a factory (e.g., an in-factory road R), a recording PC (Personal Computer) 200 for recording images captured by monitoring camera 100, and a risky behavior analysis PC 300 for analyzing risky behavior of forklifts and workers using the images recorded by recording PC 200, all connected via network N. Storage 201 for storing the recorded images is electrically connected to recording PC 200, and storage 201 for storing analysis information obtained through analysis is electrically connected to risky behavior analysis PC 300. Although FIG. 1 illustrates the recording PC 200 and risky behavior analysis PC 300 as separate computers, they may also be configured as a single computer.

[0016] Surveillance camera 100 captures images of a predetermined area within the factory and transmits the captured images to recording PC 200 via network N. Surveillance camera 100 can be implemented using a general-purpose surveillance camera capable of communications as its hardware. The predetermined area within the factory where surveillance camera 100 is installed is, for example, a location where there is a risk of collision between workers and forklifts moving about within the factory, such as an intersection, a T-junction, a backyard, or behind a container. These are merely examples, and the predetermined area within the factory may be determined according to the work environment of the factory.

[0017] The recording PC 200 is a computer that receives images captured by the surveillance camera 100 and stores them in a storage 201 in chronological order.

[0018] The risky behavior analysis PC 300 is a computer that reads the images stored in the storage 201 by the recording PC 200 and analyzes whether or not a worker or forklift has engaged in risky behavior, and what kind of risky behavior has occurred.

[0019] A general-purpose computer can be used as the hardware for the recording PC 200 and the risky behavior analysis PC 300. These computers can be realized, for example, by a general-purpose computer 1600, as shown in Figure 2 (schematic diagram of a computer), which includes a CPU 1601, a memory 1602, an external storage device 1603 such as a hard disk drive (HDD), a reading / writing device 1607 that reads and writes information from a portable storage medium 1608 such as a compact disk (CD) or USB memory, an input device 1606 that accepts input of various information such as a keyboard or mouse, an output device 1605 such as a display that outputs various input information used in processing, a communication device 1604 such as a network interface card (NIC) for connecting to a communication network, and an internal communication line (referred to as a system bus) 1609 such as a system bus that connects these devices together.

[0020] Furthermore, various data stored in these computers or used for processing can be realized by the CPU 1601 reading and using it from the memory 1602 or the external storage device 1603. Furthermore, each unit of the recording PC 200 and the risky behavior analysis PC 300 can be realized by the CPU 1601 loading a predetermined program stored in the external storage device 1603 into the memory 1602 and executing it.

[0021] The above-mentioned predetermined programs and data may be stored (downloaded) into the external storage device 1603 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and then loaded onto the memory 1602 and executed by the CPU 1601. Alternatively, the programs and data may be directly loaded onto the memory 1602 from the storage medium 1608 via the reading / writing device 1607 or from the network via the communication device 1604, and then executed by the CPU 1601.

[0022] In the following, we will explain an example in which the recording PC 200 and the risky behavior analysis PC 300 are configured by a single computer, but all or part of these functions may be distributed across one or more computers, such as a cloud, and similar functions may be realized by communicating with each other via a network.

[0023] Fig. 3 is a diagram showing an example of an environment in which control system 1000 is applied, namely, a factory. As shown in Fig. 3, the factory has a rule compliance area 310 including a crosswalk 301 for a worker to cross a travel lane 304, a worker stop line 303 for temporarily stopping a worker 302 attempting to cross the crosswalk, and a forklift stop line 306 for temporarily stopping a forklift 305 traveling on the travel lane 304. An entry stop marker 303a is provided on the near side of the worker stop line 303 in the walking direction, indicating a position where a worker should stop before entering the travel lane. Furthermore, an entry stop marker 306a is provided on the near side of the forklift stop line 306 in the traveling direction, indicating a position where a forklift should stop before entering the crosswalk.

[0024] The surveillance camera 100, installed at a predetermined position in the rule compliance area 310, captures an image of an area R within the rule compliance area 310 where danger must be avoided between the worker 302 and the forklift 305. In this example, the area R corresponds to the predetermined area within the factory. As with the predetermined area within the factory, the rule compliance area 310 may be determined according to the work environment of the factory, such as a travel route such as an intersection or T-junction, a back yard, or a place behind a container where there is a risk of collision or other danger when the worker and the forklift move relative to each other within the factory.

[0025] 3 illustrates an example of a rule-compliance area where a worker crosses a forklift path. However, the rule-compliance area is not limited to this. For example, various areas that pose a risk of collision between a worker and a forklift may be considered as rule-compliance areas, such as whether a worker stops temporarily to cross an intersection where forklift paths intersect, or whether a forklift stops temporarily to cross a path where a worker walks in a backyard for work such as unloading or loading.

[0026] 4 is a block diagram showing the functional configuration of the recording PC 200. As shown in FIG.

[0027] The image receiving unit 401 receives images captured and transmitted by the surveillance camera 100. The image storing unit 402 stores the images received by the image receiving unit 401 in the storage 201 in chronological order.

[0028] Fig. 5 is a block diagram showing the functional configuration of the risky behavior analysis PC 300. As shown in Fig. 5, the risky behavior analysis PC 300 functionally includes an image acquisition unit 501, a detection unit 502, a determination unit 503, and an output unit 504.

[0029] The image acquisition unit 501 acquires images captured by the monitoring camera 100 and stored in the recording PC 200 in real time.

[0030] The detection unit 502 detects a monitoring target for each of the images acquired by the image acquisition unit 501. Monitoring targets include forklifts traveling within a factory and workers moving around. These monitoring targets can be detected using conventionally known image AI technology based on the shape of the forklift, changes in the speed of movement, changes in the work clothes worn by the workers, changes in walking speed, etc. The detection unit 502 also tracks the monitoring targets detected for each image in chronological order.

[0031] The determination unit 503 determines whether the monitoring target tracked by the detection unit 502 is a rule determination target for determining whether or not it violates a predetermined rule. Furthermore, if the determination unit 503 determines that it violates a predetermined rule, it generates predetermined warning information for warning the forklift or worker that committed the violation, violation video information that is a video of the forklift or worker that committed the violation committing the violation, and analysis result information that indicates the result of the determination that it is a violation. On the other hand, if the determination unit 503 determines that it is not a violation of a predetermined rule, it generates analysis result information that indicates the result of the determination that it is not a violation.

[0032] The output unit 504 outputs the predetermined warning information, the violation video information, and the analysis result information generated by the determination unit 503 to the display of the risky behavior analysis PC 300. The display can be the output device 1605 shown in Fig. 2. Specific processes performed by the recording PC 200 and the risky behavior analysis PC 300 will be described later using flowcharts.

[0033] 6 is a flowchart showing the processing steps performed by this system. In the following, it is assumed that the image acquisition unit 501 of the risky behavior analysis PC 300 acquires images captured by the monitoring camera 100 from the recording PC 200 in real time.

[0034] As shown in FIG. 6, the detection unit 502 of the risky behavior analysis PC 300 detects forklifts and workers included in the rule-compliance area 310 or a predetermined area within the factory from an image, and calculates the distance between the detected forklifts and workers and a stop line located within a certain proximity (S601). More specifically, the detection unit 502 calculates the distance by calculating the length of a line connecting the center point of a rectangular area representing the forklift or worker included in the image and the closest point to the rectangular area among the areas of the stop line located a certain distance away from the rectangular area. The detection unit 502 monitors forklifts and workers whose calculated distance is within a predetermined threshold distance (e.g., 5 meters) and calculates the distance between the monitored object and the stop line. The detection unit 502 calculates the distance for each image acquired in real time.

[0035] Furthermore, the detection unit 502 compares a predetermined number of consecutive images (e.g., two) in chronological order from among the images captured in S601, tracks the target, and records a video including these images. At this time, the detection unit 502 stores in memory the time when the first frame of the predetermined number of consecutive images (e.g., two) was captured as the intrusion start time, which is the time when the target entered the rule-compliance area 310 or a predetermined area in the factory (S602). The image capture time is recorded in advance as header information of the images acquired in real time.

[0036] Fig. 7 is a diagram showing an image of the processing of S601 and S602. In Fig. 7, it is assumed that the detection unit 5902 is performing processing on a monitoring target in which the distance between the forklift or worker included in the image and the stop line is within the determination distance.

[0037] 7, in S601, the detection unit 502 calculates a distance 701a between a monitoring target (e.g., a forklift) 701 included in an image acquired at a certain point in time and a stop line 703. Furthermore, the detection unit 502 calculates a distance 701b between a monitoring target (e.g., a forklift) 702 included in an image of the next frame and the stop line 703. In this example, the detection unit 502 tracks the monitoring target by comparing the two images, and determines that the monitoring target is moving in an approach direction 704, which is the direction of the stop line 703.

[0038] At this time, the detection unit 502 compares the distance 701a with the distance 701b, and if the latter is shorter than the former by a certain amount, it predicts that the monitored object (for example, a forklift) will stop at the stop line 703. In other words, if the distance between the monitored object and the stop line in the same direction as the approach direction is shorter than a certain amount, the detection unit 502 predicts that the vehicle will brake and stop at the stop line.

[0039] Next, the determination unit 503 determines whether the monitoring target tracked by the detection unit 502 using the image is a target for determination of whether it violates a predetermined rule (S603, S604). Whether the target is a target for determination is registered in advance in the monitoring target type table.

[0040] FIG. 8 is a diagram showing an example of a monitoring object type table. The monitoring object type table is a table that determines whether a monitoring object is subject to rule violation determination. As shown in FIG. 8, the monitoring object type table 801 stores the type of monitoring object, the size of the monitoring object of that type, and a flag that determines whether the monitoring object is subject to rule violation determination, in association with each other. The type of monitoring object may be, for example, a forklift or a worker, and various objects in a factory that may be subject to monitoring in S601 may be registered. FIG. 8 shows, for example, that a forklift of size "A1" is subject to monitoring (flag = "1"), and a truck of size "A4" is not subject to monitoring (flag = "0"). In other words, trucks are subject to monitoring because their size differs by a certain amount from that of forklifts, workers, etc., and they may not fit within the above-mentioned rule compliance area 310 or a predetermined area in a factory.

[0041] When the determination unit 503 determines in S602 that the monitoring target tracked by the detection unit 502 using the images is a target for determination of whether or not it violates a predetermined rule (S604; Yes), the determination unit 503 determines whether or not the monitoring target of the type that is the target for determination is violating a rule (S605, S606). For example, when the monitoring target of the type that is the target for determination is a worker, the determination unit 503 determines that the worker is not violating a rule if, in images of consecutive frames in time series, the moving distance of the worker is less than a predetermined threshold and the change in the distance to the near side of the stop line is less than a certain threshold, and this state continues for more than a predetermined threshold time.

[0042] Fig. 9A is a diagram for explaining a method for determining whether or not a rule violation has occurred. Fig. 9A shows a case where two images 901 and 902 of consecutive frames in time series are obtained. In this example, a worker is illustrated, but the same can be applied to other monitored objects such as a forklift.

[0043] 9A, the determination unit 503 calculates a moving distance 902b, which is the distance d between the coordinates (x1, y1) of the center position of a rectangular area 901a representing the worker to be monitored that is included in an image 901 of a frame at a certain time point and the coordinates (x2, y2) of the center position of a rectangular area 902a representing the worker to be monitored that is included in an image 902 of the next frame at a certain time point. The images 901 and 902 include stop lines 901b and 902c, respectively.

[0044] The coordinates (x1, y1) and (x2, y2) of the center position are coordinates calculated by the detection unit 502 in S601. If the movement distance 902b is less than a predetermined threshold and the change in the distance to the near side of the stop line is less than a certain threshold for a predetermined threshold time or more, the determination unit 503 determines that the worker has stopped at the stop line and has not violated a rule. In FIG. 9A , if the movement distance d is less than a predetermined threshold (e.g., 1 m) and the change in the distance between the stop line 901b and the rectangular area 901a in the image 901 and the distance between the stop line 902c and the rectangular area 902a in the image 902 is less than a certain threshold (e.g., 50 cm) for a predetermined threshold time or more (e.g., 10 seconds), the determination unit 503 determines that the worker has not violated a rule. The predetermined threshold time may be calculated based on the number of frames (e.g., 3 frames). Information regarding predetermined rule violations, such as the various thresholds described above and what cases are considered to be rule violations, is determined in advance according to the monitoring location within the factory. The monitoring location includes the location where the monitoring camera 100 is actually installed, as well as the rule compliance area 310 including that location and the range of a predetermined area within the factory (see FIG. 3).

[0045] 9B is a diagram showing an example of information related to a predetermined rule violation (rule violation setting information). As shown in FIG. 9B, the rule violation setting information 911 stores a monitoring location ID for identifying the monitoring location and a rule violation criterion that serves as a criterion for determining whether or not a rule violation has occurred at the monitoring location identified by the monitoring location ID, in association with each other. FIG. 9B shows that, for example, at a monitoring location identified by the monitoring location ID "001," a rule violation is determined to have occurred if two requirements are not met: "(1) the travel distance does not meet a predetermined threshold" and "(2) a state in which the change in the distance from the near side of the stop line does not meet a certain threshold continues for a predetermined threshold time or longer."

[0046] In this way, the rule violation setting information 911 defines criteria for determining rule violations according to various monitoring positions in the factory (for example, travel routes such as intersections and T-junctions, back yards, and areas behind containers). Therefore, the administrator of this system or the factory administrator can define the criteria according to various environments within the factory (for example, the site area and width of travel routes). Furthermore, for example, if the risk level at each monitoring position is reduced by improving the environment within the factory, the criteria can be easily changed according to the degree of risk. In other words, the criteria for each monitoring position can be easily changed according to changes in the environment within the factory.

[0047] The movement distance d and the distance between the stop line and the rectangular area can be calculated by unifying the coordinate system using a conventionally known technique, such as converting the coordinates on images 901 and 902 shown in Fig. 9A into global coordinates that represent the calibration information of surveillance camera 100 and the camera position information stored in risky behavior analysis PC 300 as stop line information. The calibration information and stop line information will be described later using Figs. 11 and 12, respectively.

[0048] The determination unit 503 similarly calculates the movement distance d when, in the image of the frame next to image 902 in FIG. 9A, a rectangular area 1001a representing the worker to be monitored, included in image 1001 as shown in FIG. 10, crosses the stop line. However, in this case, the rectangular area 1001a crosses the stop line 1001b. Therefore, even if the movement distance does not meet the predetermined threshold and the state in which the change in the distance to the front side of the stop line does not meet the certain threshold continues for more than the predetermined threshold time, the determination unit 503 determines that the worker has violated the rules because he or she has not stopped at the stop line. In other words, the worker is determined to have violated the rules because he or she has crossed the stop line and is standing still.

[0049] Alternatively, by aligning the coordinate axes in images 901, 902, and 1001 with the position of the stop line, the movement distance can be calculated with the area in front of the stop line as positive and the area on the other side of the stop line as negative, and if the calculated movement distance is negative, it can be determined that the subject has crossed the stop line and stopped. In this case, even if the two requirements of "(1) the movement distance does not meet a predetermined threshold" and "(2) the state in which the change in the distance from the front side of the stop line does not meet a certain threshold continues for a predetermined threshold time or more" are met at the monitoring location identified by the monitoring location ID "001" in the rule violation setting information shown in FIG. 9B, the determination unit 503 determines that a rule violation has occurred because the additional condition of "the change in the speed of the monitored subject on the other side of the stop line is positive" is not met. In other words, even if the vehicle decelerates before the stop line and satisfies the above two requirements, if the additional condition is not met, it is determined that the vehicle decelerated according to the rules up to the stop line, but continued in this state, causing the vehicle to stop beyond the stop line (i.e., the change in speed beyond the stop line became negative), and it is determined that a violation of the rules regarding the additional condition at the monitoring position occurred. This is because, if the rules are followed, when the monitored vehicle stops, it accelerates beyond the stop line, and the change in speed should be positive. Although the above example was explained using images from two frames, images from three or more frames may also be used to determine whether the various thresholds described above are met.

[0050] If the determination unit 503 determines that the monitoring target of the type being determined has violated a rule (S606; Yes), it generates predetermined warning notification information to warn the monitoring target who has violated the rule, a violation video which is a video of the monitoring target violating the rule, and analysis result information which is the result of the determination that the violation has occurred. The output unit 504 outputs the predetermined warning notification information, the violation video, and the analysis result information to the display of the risky behavior analysis PC 300 (S607, S608, S609). For example, the determination unit 503 reads the time when the rule violation was committed, an image at that time, information indicating the rule violation, and information related to the predetermined rule violation, and generates the predetermined warning notification information. The determination unit 503 also reads the video recorded in S602 and generates the violation video. The analysis result information will be described later with reference to FIG. 13.

[0051] On the other hand, if the determination unit 503 determines that the monitoring target of the type being determined has not violated the rules (S606; No), it generates analysis result information that is the result of the determination that there is no violation, and the output unit 504 outputs the analysis result information to the display of the risky behavior analysis PC 300 (S610). The analysis result information will be described later with reference to Fig. 13. When the processing of S609 or S610 ends, the processing of this system shown in Fig. 6 ends.

[0052] Fig. 11 is a diagram showing an example of calibration information. The calibration information is a table that defines information related to the position and settings of the surveillance camera 100. As shown in Fig. 11, calibration information 1101 stores a surveillance camera ID for identifying the surveillance camera 100, a monitoring position ID for identifying the position monitored by the surveillance camera identified by the surveillance camera ID, and a calibration file that stores information related to the settings of the surveillance camera (for example, settings of resolution, angle of view, frame rate, etc.), in association with each other.

[0053] In Figure 11, for example, the surveillance camera 100 identified by the surveillance camera ID "001" is installed at the surveillance position identified by the surveillance position ID "P001", and its settings are defined in the calibration file represented by "C1".

[0054] FIG. 12 is a diagram showing an example of stop line information. The stop line information is a table that defines information related to the position and configuration of stop lines. As shown in FIG. 12, the stop line information 1201 stores the above-mentioned monitoring position ID, stop line start point, stop line end point, and entry stop position marker position in association with each other. FIG. 12 shows, for example, that the monitoring position identified by the monitoring position ID "001" includes a stop line represented by the coordinates of the stop line start point (Xa1, Ya1) and the coordinates of the stop line end point (Xa2, Ya2). It also shows that the position of the entry stop marker placed in association with the stop line is represented by the coordinates (Xa3, Ya3).

[0055] FIG. 13 is a diagram illustrating an example of analysis result information. The analysis result information is a table storing the determination result of the determination unit 503 regarding whether or not a rule has been violated and the information on which the determination was based. As illustrated in FIG. 13, the analysis result information 1301 stores, in association with each other, an analysis record ID for identifying the analysis result, the intrusion start time of the monitoring target, the monitoring location ID described above, the determination result determined by the determination unit 503 in S605 and S606 regarding whether or not a rule has been violated, the monitoring target for which the determination result was output, and the recorded video at the time of the rule violation. For example, in FIG. 13, the analysis result information identified by the analysis record ID “R001” stores the above-mentioned determination result at the monitoring location identified by the monitoring location ID “001,” and indicates that the intrusion start time, “2024 / 05 / 10 09:01:00:02,” when tracking began, is stored. The table also indicates that the determination result for the “worker” at the monitoring location was “compliant” (i.e., not violating the rules).

[0056] Furthermore, the analysis result information identified by the analysis record ID "R002" is information storing the above-mentioned determination result at the monitoring location identified by the monitoring location ID "001," which is the same as the analysis result information of the analysis record ID "R001." The information indicates that the intrusion start time, "2024 / 05 / 10 09:05:31:10," when tracking began, is stored. The information also indicates that the determination result for the object of determination, "forklift" at the monitoring location, is "violation" (i.e., a rule violation), and the video recorded at that time is stored as "YYYYYY.mp4." This video is video recorded during tracking in S602. The number of images (e.g., two) used for tracking in S602 is used as a unit to determine whether or not a specific rule has been violated in S603 and S604, and the determination of whether or not a rule violation has occurred in S605 and S606. Therefore, each record of the analysis result information 1301 is created in the above-mentioned units.

[0057] In this way, by outputting the analysis result information to the display of the risky behavior analysis PC 300, the administrator of this system or the factory administrator can easily confirm that monitored subjects who are not violating the rules are complying with the rules within the factory, while for monitored subjects who have violated the rules, they can be warned based on the above-mentioned specified warning notification information or the above-mentioned violation video in order to ensure compliance with the rules within the factory, making it possible to appropriately avoid danger to monitored subjects within the factory.

[0058] Although the present embodiment has been described above with reference to the drawings, various modifications are possible depending on the environment in which the system is used.

[0059] 6, for example, if the determination unit 503 determines that the monitoring target tracked by the detection unit 502 is not a target for determining whether or not it violates a predetermined rule (S603; No), the determination unit 503 ends the process as is. However, an image or video of the monitoring target tracked in S602 may be output to the display of the risky behavior analysis PC 300. This allows the system administrator or factory administrator to easily add a new monitoring target by checking such images and determining that even a monitoring target that is not a target for determination is likely to violate a rule by setting the flag in the monitoring target type table 801 shown in FIG. 8 to "1."

[0060] Furthermore, the determination in S604 may be performed between the worker and an intrusion stop position marker located before the stop line, rather than at the stop line. This allows a determination as to whether a rule violation has occurred even before the stop line, thereby ensuring even greater safety. For example, in FIG. 9A , if the movement distance 902b is less than a predetermined threshold and the change in the distance from the intrusion stop position markers 901c and 902d continues to be less than a certain threshold for a predetermined threshold time or longer, the determination unit 503 may determine that the worker has stopped at the stop line and has not violated the rule. However, as described in FIG. 10 , if the worker is beyond the stop line 1001b, a rule violation is determined, similar to the case of determination based on the stop line, even if the change in the distance from the intrusion stop position marker 1001c continues to be less than a certain threshold for a predetermined threshold time or longer. This is because it is clear that the worker has stopped beyond the stop line.

[0061] As described above in FIGS. 1, S605 and S606 of FIG. 6, and FIGS. 9A, 9B, 10, etc., in a control system (e.g., control system 1000) for avoiding danger to a monitored object using a computer (e.g., computer 1600) having a processor and memory, the processor reads rule violation setting information (e.g., rule violation setting information 911) that defines rules corresponding to the monitoring location of a factory, which is pre-stored in the memory, and determines that the monitored object (e.g., a forklift or worker) is violating a rule if the area in front of the stop line where the monitored object should stop and the area beyond the stop line, included in an image captured by a monitoring camera (e.g., monitoring camera 100), does not satisfy the rules defined in the rule violation setting information. This applies rules such as movement and stopping to monitored objects such as forklifts and workers according to the environment within the factory, making it possible to avoid danger more appropriately than before.

[0062] 6, S603 and S604, and FIG. 8, the processor reads a monitoring target type table (e.g., monitoring target type table 801) that is stored in advance in the memory and that defines whether the monitoring target is a target for rule violation determination, and determines whether the monitoring target is a rule violation if the image includes a monitoring target of a type defined in the monitoring target type table. This makes it possible to narrow down in advance the targets for rule violation determination, thereby enabling efficient rule violation determination.

[0063] 6, when the processor determines that the monitoring target has violated a rule, it outputs to a display device (e.g., output device 1605) of the computer predetermined warning information for warning the monitoring target who has violated the rule, violation video information of the monitoring target who has violated the rule when the monitoring target who has violated the rule committed the violation, and analysis result information (e.g., analysis result information 1301) showing the result of the determination that the monitoring target has violated the rule. This allows the system administrator or factory administrator to warn the monitoring target who has violated the rule based on the predetermined warning notification information and the violation video in order to have them comply with the rules in the factory, and makes it possible to appropriately avoid danger to the monitoring target in the factory.

[0064] 6, if the image does not include a monitoring target of a type defined in the monitoring target type table, the processor outputs the image to a display device (e.g., output device 1605) of the computer. This allows the system administrator or factory administrator to check the image and, if they determine that a monitoring target that does not fall under the judgment target category is highly likely to violate a rule, they can easily add a new monitoring target to the monitoring target type table 801 as needed.

[0065] 3, 9, 10, etc., the processor determines that the monitoring target is violating a rule when the monitoring target does not satisfy the rule defined in the rule violation setting information with respect to the entry stop position marker provided in the near area (for example, entry stop position markers 901c, 902d). This allows a determination as to whether or not a rule violation has occurred even before the stop line, thereby ensuring even greater safety.

[0066] In this way, according to this embodiment, it is possible to provide support for reliably ensuring the safety of monitored objects in a factory, such as forklifts and workers, thereby providing a safer, more comfortable working environment with fewer risks in the factory.

[0067] The present invention is not limited to the above-described embodiments as they are, and in the implementation stage, the components can be modified and embodied within the scope of the gist of the present invention, or multiple components disclosed in the above-described embodiments can be appropriately combined. [Explanation of symbols]

[0068] 1000 Control System 100 surveillance cameras 200 Recording PC 300 PC for risky behavior analysis N Network 401 Image receiving unit 402 Image storage section 501 Image acquisition unit 502 Detection unit 503 Judgment section 504 Output Section

Claims

1. A control system for avoiding danger to a monitored object by a computer having a processor and a memory, comprising: The processor: reading out rule violation setting information that defines rules according to monitoring positions in the factory, which is stored in advance in the memory; When the area in front of the stop line where the monitoring target should stop and the area on the other side of the stop line, which are included in the image at the monitoring position of the factory captured by the monitoring camera, do not satisfy the rules defined in the rule violation setting information, the monitoring target is determined to be violating the rules. A control system comprising:

2. The processor: reading a monitoring target classification table stored in advance in the memory, which defines whether the monitoring target is a target for rule violation determination; determining whether the image violates the rule when a monitoring target of a type defined in the monitoring target type table is included in the image; 2. The control system of claim 1.

3. The processor: When it is determined that the monitoring subject has violated a rule, the computer outputs to a display device thereof predetermined warning information for warning the monitoring subject who has violated the rule, violation video information of the monitoring subject who has violated the rule when the monitoring subject committed the violation, and analysis result information indicating the result of the determination that the rule has been violated.

2. The control system of claim 1.

4. The processor: If the image does not include a monitoring target of a type defined in the monitoring target type table, output the image to a display device of the computer.

3. The control system of claim 2.

5. The processor: When the monitoring target does not satisfy the rule defined in the rule violation setting information with respect to the intrusion stop position marker provided in the near side area, the monitoring target is determined to be violating the rule.

2. The control system of claim 1.

6. A control method for avoiding danger to a monitored object, which is performed by a computer, comprising: Reads out rule violation setting information that has been stored in advance in memory and defines rules according to the monitoring location in the factory, When the area in front of the stop line where the monitoring target should stop and the area on the other side of the stop line, which are included in the image at the monitoring position of the factory captured by the monitoring camera, do not satisfy the rules defined in the rule violation setting information, the monitoring target is determined to be violating the rules. A control method comprising:

7. reading a monitoring target classification table stored in advance in the memory, which defines whether the monitoring target is a target for rule violation determination; determining whether the image violates the rule when a monitoring target of a type defined in the monitoring target type table is included in the image; 7. The control method according to claim 6.

8. When it is determined that the monitoring subject has violated a rule, the computer outputs to a display device thereof predetermined warning information for warning the monitoring subject who has violated the rule, violation video information of the monitoring subject who has violated the rule when the monitoring subject committed the violation, and analysis result information indicating the result of the determination that the rule has been violated.

7. The control method according to claim 6.

9. If the image does not include a monitoring target of a type defined in the monitoring target type table, output the image to a display device of the computer.

8. The control method according to claim 7.

10. When the monitoring target does not satisfy the rule defined in the rule violation setting information with respect to the intrusion stop position marker provided in the near side area, the monitoring target is determined to be violating the rule.

7. The control method according to claim 6.

Images