Monitoring device, monitoring system, and fuel permit issuing terminal

Abstract

To provide a monitoring device that can confirm safety more surely for fueling action by users themselves of a fueling station.SOLUTION: A monitoring device 22 communicatively connects to a fueling permission issuing terminal 24 that accepts an operation for permitting fueling action by a fueling worker. The monitoring device 22 receives a captured image T from an imaging device 26 that captures an image of surroundings of a weighing machine 4 installed at a fueling station SS. The monitoring device 22 extracts confirmation required areas DA1 to DA3 from the captured image T. The confirmation required areas DA1 to DA3 are areas necessary to be confirmed in order to permit fueling action by the fueling worker. The monitoring device 22 generates a display image and transmits the display image to the fueling permission issuing terminal 24. The display image includes the captured image T and a presentation image PR1. The presentation image PR1 is an image that presents the confirmation required areas DA1 to DA3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a monitoring device, a monitoring system, and a fuel supply permission issuing terminal. [Background technology]

[0002] One type of gas station is the self-service gas station. Self-service gas stations are called "self-service stations." At self-service stations, customers fill up their gas tanks themselves. Customers are permitted to fill up their gas tanks under the supervision of a hazardous materials handler.

[0003] More specifically, a self-service station is equipped with a monitoring room, and a self-service console (SSC) is installed inside the monitoring room. The staff at the self-service station directly and visually monitors the customers at the station from the monitoring room, and when they confirm that it is safe, they press the refueling permission switch on the SSC. As a result, the customer is permitted to refuel. Alternatively, a surveillance camera that captures images of the area around the weighing machine may be installed at the self-service station. In this case, the staff at the self-service station monitors the customers at the station via images captured by the surveillance camera.

[0004] Patent Document 1 discloses a handheld transceiver equipped with a monitor and a refueling permission switch. The handheld transceiver can be carried by a self-service station employee. Images captured by a surveillance camera are displayed on the monitor of the handheld transceiver. According to the handheld transceiver of Patent Document 1, an employee can monitor a customer via the monitor of the handheld transceiver, and when safety is confirmed, the employee can press the refueling permission switch of the handheld transceiver to permit the customer to refuel. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2002-193399 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, simply having employees of a self-service gas station directly monitor customers from a monitoring room with their own eyes or through images captured by a surveillance camera does not mean that there is a possibility that a customer may be allowed to refuel even if a safety check is overlooked.

[0007] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a monitoring device, a monitoring system, and a refueling permission issuing terminal that can more reliably confirm the safety of refueling actions by customers at refueling stations themselves. [Means for solving the problem]

[0008] The monitoring device according to the present invention is communicatively connected to a refueling permission issuing terminal that accepts operations for authorizing a refueling operation by a refueling worker. The monitoring device includes an image receiving unit, an area extraction unit, an image generating unit, and a display image transmitting unit. The image receiving unit receives an image from an imaging device that captures an image of the periphery of a weighing machine installed at a refueling station. The area extraction unit extracts, from the image, an area requiring confirmation that needs to be confirmed in order to authorize the refueling operation by the refueling worker. The image generating unit generates a display image including the image and a presentation image that presents the area requiring confirmation. The display image transmitting unit transmits the display image to the refueling permission issuing terminal.

[0009] The monitoring system according to the present invention includes a monitoring device and an imaging device. The monitoring device is communicatively connected to a refueling permission issuing terminal that accepts operations for permitting a refueling action by a refueling worker. The imaging device captures an image of the periphery of a weighing machine installed at a refueling station. The monitoring device includes an image receiving unit, an area extracting unit, an image generating unit, and a display image transmitting unit. The image receiving unit receives an image from the imaging device. The area extracting unit extracts a confirmation-required area from the captured image that needs to be confirmed in order to permit the refueling action by the refueling worker. The image generating unit generates a display image including the captured image and a presentation image that presents the confirmation-required area. The display image transmitting unit transmits the display image to the refueling permission issuing terminal.

[0010] The above monitoring system may further include the refueling permit issuing terminal.

[0011] Another monitoring system according to the present invention includes a refueling permission issuing terminal and a monitoring device. The refueling permission issuing terminal accepts an operation for permitting a refueling action by a refueling worker. The monitoring device is communicably connected to the refueling permission issuing terminal. The monitoring device includes an image receiving unit, an area extracting unit, an image generating unit, and a display image transmitting unit. The image receiving unit receives an image from an imaging device that captures an image of the periphery of a weighing machine installed at a refueling station. The area extracting unit extracts a confirmation-required area from the image, which needs to be confirmed in order to permit the refueling action by the refueling worker. The image generating unit generates a display image including the image and a presentation image presenting the confirmation-required area. The display image transmitting unit transmits the display image to the refueling permission issuing terminal.

[0012] The refueling permission issuing terminal according to the present invention includes a display image receiving unit, a display unit, a decision unit, and a first operation acceptance unit. The display image receiving unit receives a display image from a monitoring device, the display image including an image acquired from an imaging device that takes an image of the periphery of a weighing machine installed at a refueling station, and a presentation image that presents a confirmation-required area that needs to be confirmed in order to permit the refueling action by the refueling worker. The display unit displays the display image. The decision unit decides whether or not to enable a refueling permission operation for permitting the refueling action by the refueling worker. The first operation acceptance unit accepts the refueling permission operation by an operator. Effect of the Invention

[0013] The monitoring device, monitoring system, and refueling permission issuing terminal according to the present invention can more reliably confirm the safety of refueling actions performed by customers at refueling stations themselves. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 illustrates a gas station system. [Diagram 2] FIG. [Diagram 3] FIG. [Figure 4] 1 is a block diagram showing a configuration of a video processing server according to a first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram showing an example of a captured image. [Figure 6] FIG. 1 is a block diagram showing a configuration of a mobile terminal according to a first embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing a screen displayed on the mobile terminal according to the first embodiment of the present invention. [Figure 8] FIG. 2 is a diagram showing a screen displayed on the mobile terminal according to the first embodiment of the present invention. [Figure 9] FIG. 2 is a diagram showing a screen displayed on the mobile terminal according to the first embodiment of the present invention. [Figure 10] 5 is a flowchart showing a process executed by a processing unit of the video processing server according to the first embodiment of the present invention. [Figure 11] 5 is a flowchart showing a process executed by a terminal processing unit of the mobile terminal according to the first embodiment of the present invention. [Figure 12] FIG. 11 is a diagram showing a screen displayed on a mobile terminal according to a second embodiment of the present invention. [Figure 13] 10 is a flowchart showing a process executed by a processing unit of a video processing server according to a second embodiment of the present invention. [Figure 14] 10 is a flowchart showing a process executed by a terminal processing unit of a mobile terminal according to a second embodiment of the present invention. [Figure 15] FIG. 11 is a diagram showing a screen displayed on a mobile terminal according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, with reference to the drawings (FIGS. 1 to 15), an embodiment of the monitoring device, monitoring system, and fueling permission issuing terminal of the present invention will be described. However, the present invention is not limited to the following embodiment, and can be embodied in various aspects without departing from the gist of the present invention. Note that where explanations overlap, explanations may be omitted as appropriate. Also, in the drawings, the same or equivalent parts are given the same reference symbols, and the same explanations will not be repeated.

[0016] [Embodiment 1] A first embodiment of the present invention will be described with reference to Figs. 1 to 11. First, a gas station system 100 will be described with reference to Fig. 1. Fig. 1 is a diagram showing the gas station system 100. The gas station system 100 is applied to a self-service gas station (a self-service gas station). The gas station system 100 is a system that allows customers of the self-service gas station to refuel themselves. Hereinafter, a customer of the self-service gas station may be referred to as a "customer". A customer is, for example, a passenger in a vehicle C (fuel target) such as a passenger car or a motorcycle.

[0017] As shown in Fig. 1, the gas station system 100 includes a monitoring system 2, a weighing machine 4, an SSC (self-service console) 6, and a POS (point of sale) 8. The weighing machine 4 is installed on the premises of the self-service stand SS. The SSC 6 and the POS 8 are installed in a monitoring room of the self-service stand SS. Although there is only one weighing machine 4 in this embodiment, the gas station system 100 may include multiple weighing machines 4.

[0018] The metering machine 4 is a facility (fuel supply facility) for a customer to supply fuel. The metering machine 4 includes a fuel supply system and a control unit that controls the fuel supply system. The fuel supply system includes a fuel supply pump and a flow meter. The metering machine 4 includes a fuel supply nozzle 41 (see FIG. 2), and fuel is supplied to the vehicle C from the fuel supply nozzle 41 by operating the fuel supply pump.

[0019] The weighing machine 4 is connected to the SSC 6 via wire or wirelessly so as to be able to communicate with the SSC 6. The weighing machine 4 accepts the customer's selection of the type of oil, the selection of the payment method, and the selection of the amount of fuel to be supplied to the vehicle C or the payment amount. The weighing machine 4 transmits data to the SSC 6 indicating the type of oil, the payment method, and the amount of fuel to be supplied or the payment amount selected by the customer.

[0020] The POS 8 is connected to the SSC 6 by wire or wirelessly so as to be able to communicate with the SSC 6. The POS 8 collects data indicating the type of fuel selected by the customer, the payment method, and the amount of fuel dispensed or the payment amount via the SSC 6, and performs settlement processing, sales management, etc. In this embodiment, the gas station system 100 includes the SSC 6 and the POS 8, but the gas station system 100 may also include a POS that includes the SSC 6.

[0021] The monitoring system 2 is a system for monitoring the refueling actions of customers (refueling workers). As shown in Fig. 1, the monitoring system 2 includes a video processing server 22, a plurality of mobile terminals 24, and a plurality of imaging devices 26. The video processing server 22 is an example of a monitoring device, and the mobile terminal 24 is an example of a refueling permission issuing terminal.

[0022] The imaging device 26 captures an image of the periphery of the weighing machine 4 to generate a captured image T. The imaging device 26 is communicatively connected to the image processing server 22. For example, the imaging device 26 may be communicatively connected to the image processing server 22 via a public communication network such as the Internet. Alternatively, the imaging device 26 may be communicatively connected to the image processing server 22 by wire or wirelessly. The imaging device 26 transmits the captured image T to the image processing server 22.

[0023] In this embodiment, the multiple imaging devices 26 include a first imaging device 26a and a second imaging device 26b. However, the number of imaging devices 26 is not limited to two. The surveillance system 2 may include three or more imaging devices 26. Alternatively, the number of imaging devices 26 may be one.

[0024] The image processing server 22 generates a display image to be displayed on the mobile terminal 24 based on the captured image T (an image showing the periphery of the weighing machine 4). The image processing server 22 further extracts a confirmation-required area from the captured image T. The confirmation-required area is an area that needs to be confirmed in order to permit the customer to fill up with fuel. The display image includes the captured image T and a first presentation image PR1. The first presentation image PR1 is an image that presents the confirmation-required area in the captured image T.

[0025] The areas to be confirmed include, for example, a vehicle area, a person area, and a nozzle area. The vehicle area is an area where the vehicle C to be refueled is parked. The person area is an area where a customer performs the act of refueling. The nozzle area is an area where the nozzle portion of the refueling nozzle 41 (see FIG. 2) is inserted into the fuel filler opening of the vehicle C. The areas to be confirmed are determined in advance.

[0026] The mobile terminal 24 is a terminal carried and used by an employee of the self-service station SS. Hereinafter, an employee of the self-service station SS may be referred to as an "employee." The employee can operate the mobile terminal 24 to permit a customer to refuel. In other words, the mobile terminal 24 accepts an operation to permit a customer to refuel. Hereinafter, an operation to permit a customer to refuel may be referred to as a "refueling permission operation."

[0027] The mobile terminal 24 is communicatively connected to the video processing server 22. For example, the mobile terminal 24 may be communicatively connected to the video processing server 22 via a public communication network such as the Internet. Alternatively, the mobile terminal 24 may be communicatively connected to the video processing server 22 wirelessly.

[0028] The mobile terminal 24 receives the display image (the captured image T and the first presentation image PR1) from the video processing server 22. The mobile terminal 24 displays the display image received from the video processing server 22.

[0029] In this embodiment, the mobile terminal 24 displays the first presentation image PR1 (an image presenting a check-required area in the captured image T). Therefore, the check-required area can be presented to the staff, making it difficult for the staff to overlook the check-required area. As a result, the safety of the customer's own fueling action can be more reliably confirmed.

[0030] The mobile terminal 24 is wirelessly connected to be able to communicate with the SSC 6. When the mobile terminal 24 receives a refueling permission operation, it transmits a refueling permission signal to the SSC 6. The refueling permission signal is a signal indicating that the refueling action is permitted.

[0031] 1, the monitoring system 2 includes two mobile terminals 24, but the number of mobile terminals 24 is not limited to two. The monitoring system 2 may include three or more mobile terminals 24. Alternatively, the number of mobile terminals 24 may be one.

[0032] The SSC 6 monitors the operation status of the metering machine 4. In addition, the SSC 6 permits the customer to refuel in response to receiving a refueling permission signal from the mobile terminal 24. In other words, the customer cannot refuel until the SSC 6 permits the customer to refuel.

[0033] In detail, the SSC 6 transmits a drive permission signal to the metering machine 4 in response to receiving a fuel supply permission signal from the mobile terminal 24. The drive permission signal is a signal that permits the drive of the fuel supply system of the metering machine 4. Specifically, the drive permission signal permits the operation of the fuel supply pump. Therefore, the customer can perform the act of fuel supply after the fuel supply permission signal is transmitted from the mobile terminal 24 to the SSC 6 and the drive permission signal is transmitted from the SSC 6 to the metering machine 4.

[0034] Next, the gas station system 100 will be further described with reference to Fig. 1. The metering machine 4 transmits a fuel supply permission request signal to the SSC 6 when operated by a customer. The fuel supply permission request signal is a signal requesting permission for the customer to supply fuel. In detail, the metering machine 4 transmits the fuel supply permission request signal to the SSC 6 in response to receiving an operation to select the type of fuel, an operation to select the payment method, and an operation to select the amount of fuel to be supplied or the payment amount.

[0035] In response to receiving a refueling permission request signal from the metering machine 4, the SSC 6 transmits a monitoring start signal to the portable terminal 24. In response to receiving the monitoring start signal, the portable terminal 24 transmits a request signal requesting a display image to the image processing server 22. In response to receiving the request signal, the image processing server 22 transmits (distributes) the display image (captured image T and first presentation image PR1) to the multiple portable terminals 24. As a result, the multiple portable terminals 24 display the display image.

[0036] When the mobile terminal 24 accepts an operation performed by an employee on the display image, it transmits operation information indicating the content of the operation performed on the display image to the video processing server 22. The video processing server 22 determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image T based on the operation information received from the mobile terminal 24. If a predetermined operation has been performed on the confirmation-required area in the captured image T, the video processing server 22 executes a fixation process. The fixation process will be described later with reference to FIG. 4.

[0037] Next, the weighing machine 4 will be described with reference to Fig. 2. Fig. 2 is a front view of the weighing machine 4. As shown in Fig. 2, the weighing machine 4 includes fuel supply nozzles 41a, 41b, 41c, fuel supply hoses 42a, 42b, 42c, nozzle hangers 43a, 43b, 43c, a touch display 44, and a static electricity removing sheet 45. The fuel supply nozzles 41a, 41b, 41c, the fuel supply hoses 42a, 42b, 42c, the nozzle hangers 43a, 43b, 43c, the touch display 44, and the static electricity removing sheet 45 are provided on the front surface 4a of the weighing machine 4.

[0038] The metering machine 4 is provided with a fuel supply system for each of the three types of fuel, regular gasoline, high-octane gasoline, and diesel, in order to selectively supply the three types of fuel, regular gasoline, high-octane gasoline, and diesel, to the vehicle C.

[0039] One end of refueling hose 42a is connected to a refueling system for regular gasoline. The other end of refueling hose 42a is connected to refueling nozzle 41a. Similarly, one end of refueling hose 42b is connected to a refueling system for premium gasoline. The other end of refueling hose 42b is connected to refueling nozzle 41b. One end of refueling hose 42c is connected to a refueling system for diesel. The other end of refueling hose 42c is connected to refueling nozzle 41c.

[0040] When the fuel nozzle 41a is not being used for fueling, it is hung on the nozzle hanger 43a. Similarly, when the fuel nozzle 41b is not being used for fueling, it is hung on the nozzle hanger 43b. When the fuel nozzle 41c is not being used for fueling, it is hung on the nozzle hanger 43c.

[0041] Hereinafter, when it is not necessary to distinguish between the fuel nozzles 41a to 41c, the fuel nozzles 41a to 41c may be referred to as "fuel nozzles 41". Similarly, when it is not necessary to distinguish between the fuel hoses 42a to 42c, the fuel hoses 42a to 42c may be referred to as "fuel hose 42". When it is not necessary to distinguish between the nozzle hangers 43a to 43c, the nozzle hangers 43a to 43c may be referred to as "nozzle hangers 43". In addition, when it is not necessary to distinguish between the regular gasoline fuel system, the high-octane gasoline fuel system, and the diesel fuel system, the regular gasoline fuel system, the high-octane gasoline fuel system, and the diesel fuel system may be referred to as "fuel systems".

[0042] When refueling the vehicle C, the customer inserts the nozzle portion of the fuel nozzle 41 into the fuel inlet of the vehicle C and then pulls the lever of the fuel nozzle 41. At this time, if the metering machine 4 has already received the drive permission signal described with reference to FIG. 1 from the SSC 6, the fuel pump is operated and fuel is supplied from the fuel nozzle 41 to the fuel tank of the vehicle C. For example, when refueling the vehicle C with regular gasoline, the customer inserts the nozzle portion of the fuel nozzle 41a into the fuel inlet of the vehicle C and then pulls the lever of the fuel nozzle 41a. At this time, if the metering machine 4 has already received the drive permission signal from the SSC 6, the control unit of the metering machine 4 operates the fuel pump provided in the fuel supply system for regular gasoline. As a result, regular gasoline is supplied from the fuel nozzle 41a to the fuel tank of the vehicle C.

[0043] The touch display 44 accepts touch operations by the customer. The touch operation is, for example, a single tap. For example, the touch display 44 sequentially displays an oil type selection screen, a payment method selection screen, and a fuel amount or payment amount selection screen in response to the touch operation by the customer.

[0044] The oil type selection screen is a screen that allows the customer to select one of the oil types: regular gasoline, high-octane gasoline, and diesel. When the touch display 44 displays the oil type selection screen, the customer can select one of the oil types: regular gasoline, high-octane gasoline, and diesel by a touch operation (oil type selection operation).

[0045] The payment method selection screen is a screen that allows a customer to select a payment method. For example, the payment method selection screen allows a customer to select whether to pay by cash or by credit card. When the touch display 44 displays the payment method selection screen, the customer can select whether to pay by cash or by credit card by a touch operation (payment method selection operation).

[0046] The fuel quantity or payment amount selection screen is a screen that allows the customer to select the fuel quantity or payment amount. When the touch display 44 displays the fuel quantity or payment amount selection screen, the customer can select the fuel quantity or payment amount by a touch operation (fuel quantity or payment amount selection operation).

[0047] In response to a touch operation by the customer, the metering machine 4 transmits a fuel supply permission request signal to the SSC 6, as described with reference to Fig. 1. Specifically, the metering machine 4 transmits a fuel supply permission request signal to the SSC 6 after the customer selects, by a touch operation, one of regular gasoline, premium gasoline, and diesel, selects by a touch operation whether to pay by cash or credit card, and selects by a touch operation the amount of fuel to be supplied or the payment amount.

[0048] Static electricity removal sheet 45 removes static electricity that has been charged to a customer (a fuel pump worker). Specifically, the static electricity that has been charged to the customer is removed when the customer touches static electricity removal sheet 45 with his or her hand. After touching static electricity removal sheet 45 with his or her hand, the customer pumps fuel.

[0049] The oil type selection operation, the payment method selection operation, and the fuel amount or payment amount selection operation may be performed on one screen. Alternatively, any two of the oil type selection operation, the payment method selection operation, and the fuel amount or payment amount selection operation may be performed on one screen.

[0050] Next, the self-service fuel station SS will be described with reference to Fig. 3. Fig. 3 is a diagram showing the self-service fuel station SS. As shown in Fig. 3, the self-service fuel station SS is provided with a plurality of fuel lanes L and an island LA.

[0051] The island LA is raised from the site of the self-service station SS. The weighing machine 4 is installed on the island LA. Fuel supply lanes L are provided on both sides of the island LA. Each fuel supply lane L includes a fuel supply area P. The fuel supply area P is adjacent to the weighing machine 4.

[0052] A customer at a self-service fuel station SS drives a vehicle C into one of a number of fuel lanes L and then parks the vehicle in a fueling area P. In this embodiment, a number of frame lines FL are drawn on the premises of the self-service fuel station SS. More specifically, a frame line FL is drawn for each fueling area P. A customer parks the vehicle C within the frame line FL and then refuels the vehicle C. Note that the frame line FL may be omitted.

[0053] In this embodiment, the multiple refueling lanes L include a first refueling lane L1 and a second refueling lane L2. The first imaging device 26a shown in Fig. 1 images the refueling area P of the first refueling lane L1. The second imaging device 26b shown in Fig. 1 images the refueling area P of the second refueling lane L2.

[0054] Hereinafter, the refueling area P of the first refueling lane L1 may be referred to as the "first refueling area P1." Similarly, the refueling area P of the second refueling lane L2 may be referred to as the "second refueling area P2." Furthermore, the captured image T transmitted from the first imaging device 26a to the video processing server 22 may be referred to as the "first captured image T1." Similarly, the captured image T transmitted from the second imaging device 26b to the video processing server 22 may be referred to as the "second captured image T2." The first captured image T1 indicates the first refueling area P1, and the second captured image T2 indicates the second refueling area P2.

[0055] Next, the weighing machine 4 will be further described with reference to Fig. 3. As shown in Fig. 3, the weighing machine 4 has a front surface 4a and a rear surface 4b. The front surface 4a of the weighing machine 4 is the surface facing the first fuel supply lane L1 (first fuel supply area P1), and the rear surface 4b of the weighing machine 4 is the surface facing the second fuel supply lane L2 (second fuel supply area P2). In this embodiment, the weighing machine 4 further includes a fuel supply nozzle 41, a fuel supply hose 42, a nozzle hanger 43, a touch display 44, and a static electricity removing sheet 45 provided on the rear surface 4b.

[0056] 3, one island LA is provided in the self-service station SS, but multiple islands LA may be provided in the self-service station SS. The number of weighing machines 4, the number of fuel lanes L, and the number of imaging devices 26 increase as the number of islands LA increases.

[0057] Next, the configuration of the video processing server 22 of this embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the video processing server 22 of this embodiment. As shown in Fig. 4, the video processing server 22 includes a communication unit 221, a processing unit 222, and a storage unit 223. The communication unit 221 is an example of a captured image receiving unit, a display image transmitting unit, an operation information receiving unit, and a request signal receiving unit.

[0058] The communication unit 221 controls communication between the multiple mobile terminals 24 and the multiple imaging devices 26 described with reference to Fig. 1. The communication unit 221 has, for example, a LAN (Local Area Network) board or a LAN module. Specifically, the communication unit 221 receives a captured image T from the imaging device 26. The communication unit 221 receives a request signal described with reference to Fig. 1 from the mobile terminal 24. The communication unit 221 transmits a display image (captured image T and first presentation image PR1) described with reference to Fig. 1 to the mobile terminal 24. The communication unit 221 receives operation information described with reference to Fig. 1 from the mobile terminal 24.

[0059] The storage unit 223 includes, for example, a semiconductor memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and a large-capacity storage medium such as a HDD (Hard Disk Drive). The storage unit 223 stores various computer programs executed by the processing unit 222. The storage unit 223 also stores various types of data. The storage unit 223 saves the first captured image T1 and the second captured image T2 received by the communication unit 221 from the first imaging device 26a and the second imaging device 26b.

[0060] The processing unit 222 executes various types of processing. The processing unit 222 has a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor executes a computer program stored in the storage unit 223 to execute various types of processing. The processing unit 222 may further have an integrated circuit dedicated to image processing. The integrated circuit dedicated to image processing may be an integrated circuit such as an ASIC (Application Specific Integrated Circuit). The processing unit 222 is an example of a region extraction unit.

[0061] The processing unit 222 extracts the confirmation-required region described with reference to Fig. 1 from the captured image T stored in the storage unit 223. The processing unit 222 may extract the confirmation-required region by, for example, pattern matching. Alternatively, the processing unit 222 may extract the confirmation-required region by using a trained model. In other words, the processing unit 222 may extract the confirmation-required region by AI (Artificial Intelligence).

[0062] When extracting a check-required region by pattern matching, the storage unit 223 stores a plurality of reference images in advance. The processing unit 222 identifies a check-required region in the captured image T by executing pattern matching between the captured image T and the plurality of reference images.

[0063] When extracting a region to be confirmed by AI, the storage unit 223 stores a trained model in advance. The processing unit 222 inputs the captured image T to the trained model. As a result, the region to be confirmed in the captured image T is identified by the trained model. When extracting a region to be confirmed by AI, the processing unit 222 may further include an NCU (Neural Network Processing Unit).

[0064] The trained model is generated by executing a machine learning algorithm. The machine learning algorithm is, for example, any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning. For example, the trained model includes a decision tree, a nearest neighbor method, a naive Bayes classifier, a support vector machine, or a neural network. For example, the neural network includes an input layer, one or more intermediate layers, and an output layer. Specifically, the neural network is a deep neural network (DNN), a recurrent neural network (RNN), or a convolutional neural network (CNN).

[0065] When the processing unit 222 receives a request signal transmitted from the mobile terminal 24 via the communication unit 221, the processing unit 222 generates a display image. Then, the communication unit 221 is controlled to transmit the display image (the captured image T and the first presentation image PR1) to the mobile terminal 24. The first presentation image PR1 includes, for example, a character image indicating the confirmation-required area. Alternatively, the first presentation image PR1 may include a mark image indicating the confirmation-required area. The processing unit 222 is an example of an image generation unit.

[0066] In detail, the SSC 6 transmits a monitoring start signal indicating the identification information of the refueling lane L to be monitored to the mobile terminal 24. The mobile terminal 24, which has received the monitoring start signal, transmits a request signal indicating the identification information of the refueling lane L to be monitored to the video processing server 22. The processing unit 222, which has received the request signal, generates a display image including a captured image T corresponding to the refueling lane L to be monitored. Then, the processing unit 222 controls the communication unit 221 to transmit the display image to multiple mobile terminals 24. Hereinafter, the refueling lane L to be monitored may be referred to as a "target lane LT."

[0067] In this embodiment, the monitoring start signal indicates the identification information of the first fuel lane L1 or the second fuel lane L2. When a customer touches the touch display 44 provided on the front surface 4a of the weighing machine 4, the weighing machine 4 transmits a fuel supply permission request signal indicating the identification information of the first fuel lane L1 to the SSC 6. As a result, a monitoring start signal indicating the identification information of the first fuel lane L1 is transmitted from the SSC 6 to the mobile terminal 24, and a request signal indicating the identification information of the first fuel lane L1 is transmitted from the mobile terminal 24 to the image processing server 22. Similarly, when a customer touches the touch display 44 provided on the back surface 4b of the weighing machine 4, the weighing machine 4 transmits a fuel supply permission request signal indicating the identification information of the second fuel lane L2 to the SSC 6. As a result, a monitoring start signal indicating the identification information of the second fuel lane L2 is transmitted from the SSC 6 to the mobile terminal 24, and a request signal indicating the identification information of the second fuel lane L2 is transmitted from the mobile terminal 24 to the image processing server 22.

[0068] When the processing unit 222 receives a request signal indicating the identification information of the first refueling lane L1, it generates a display image including the first captured image T1. Similarly, when the processing unit 222 receives a request signal indicating the identification information of the second refueling lane L2, it generates a display image including the second captured image T2.

[0069] In this embodiment, the processing unit 222 further executes a first determination process, a second determination process, and a fixed process. The first determination process, the second determination process, and the fixed process will be described below. The processing unit 222 is an example of a determination result information generating unit.

[0070] <First Judgment Process> The first judgment process is a process for judging whether or not the image of the confirmation-required area is an image that matches the refueling non-permission condition. The refueling non-permission condition is a condition that does not permit a customer to refuel. When the processing unit 222 receives a request signal via the communication unit 221, the processing unit 222 executes the first judgment process based on the captured image T stored in the memory unit 223. The processing unit 222 executes the first judgment process by, for example, pattern matching. Alternatively, the processing unit 222 executes the first judgment process by using a trained model. Hereinafter, the judgment result of the first judgment process may be referred to as the "first judgment result."

[0071] The first judgment process is executed for each area to be confirmed. The first judgment result indicates whether or not an image of the area to be confirmed is an image that matches the refueling non-permission condition. Therefore, the first judgment result indicates whether or not a situation has occurred in the area to be confirmed that makes it impossible to permit refueling. The processing unit 222 generates judgment result information based on the first judgment result. Then, the processing unit 222 controls the communication unit 221 to transmit the judgment result information to the mobile terminal 24. The communication unit 221 is an example of a judgment result information transmission unit. The judgment result information indicates whether or not a situation has occurred in the area to be confirmed that makes it impossible to permit refueling, for each area to be confirmed. In other words, the judgment result information indicates whether or not the refueling may be permitted, for each area to be confirmed. The judgment result information is, for example, a flag.

[0072] Hereinafter, among the areas requiring confirmation, areas where a situation has occurred that makes refueling actions impermissible may be referred to as "NG areas."Furthermore, among the areas requiring confirmation, areas where a situation has not occurred that makes refueling actions impermissible may be referred to as "OK areas."

[0073] <Second judgment process and fixing process> As described with reference to Fig. 1, the mobile terminal 24 transmits operation information (information indicating the content of an operation performed on a display image) to the video processing server 22. Upon receiving the operation information via the communication unit 221, the processing unit 222 executes a second determination process based on the operation information. The second determination process is a process for determining whether or not a predetermined operation has been performed on a confirmation-required area in the captured image T. Hereinafter, the determination result of the second determination process may be referred to as a "second determination result."

[0074] The processing unit 222 executes the fixing process based on the second judgment result. In detail, when a predetermined operation is performed on the confirmation-required area in the captured image T, the second judgment result indicates that the confirmation-required area in which the predetermined operation is performed has been confirmed. When the processing unit 222 determines that a predetermined operation is performed on the confirmation-required area in the captured image T, the processing unit 222 fixes the judgment result for the confirmation-required area in which the predetermined operation is performed in the judgment result information to a judgment result indicating that the refueling action may be permitted. Therefore, the judgment result information after the fixing process indicates that the confirmation-required area in which the predetermined operation is performed is an OK area. The processing unit 222 controls the communication unit 221 to transmit the judgment result information after the fixing process to the mobile terminal 24.

[0075] Next, the confirmation-required areas will be described with reference to Figs. 4 and 5. Fig. 5 is a diagram showing an example of a captured image T. As shown in Fig. 5, the processing unit 222 identifies, for example, confirmation-required areas DA1 to DA3 in the captured image T. In this embodiment, the confirmation-required area DA1 is a vehicle area. The confirmation-required area DA2 is a person area. The confirmation-required area DA3 is a nozzle area. Hereinafter, the confirmation-required area DA1 may be referred to as the "vehicle area DA1". Similarly, the confirmation-required area DA2 may be referred to as the "person area DA2". The confirmation-required area DA3 may be referred to as the "nozzle area DA3". Here, the refueling non-permission conditions for each of the confirmation-required areas DA1 to DA3 will be described.

[0076] The refueling non-permission conditions for the vehicle area DA1 include, for example, that the vehicle C is not parked in the refueling area P, that the door of the vehicle C is open, or that the window of the vehicle C is open. When the vehicle C is not parked in the refueling area P, that the door of the vehicle C is open, or that the window of the vehicle C is open, the first judgment result indicates that the vehicle area DA1 is an "NG area". In other words, the first judgment result indicates that a situation has occurred in which refueling cannot be permitted in the vehicle area DA1. Here, the fact that the vehicle C is not parked in the refueling area P includes, in addition to the fact that the vehicle C is not parked in the refueling area P, that there is a refueling target other than the vehicle C in the refueling area P. The refueling target other than the vehicle C is, for example, a portable can or a plastic container.

[0077] The conditions for not allowing refueling in the person area DA2 include, for example, that person A present in the person area DA2 is engaging in inappropriate behavior. When person A is engaging in inappropriate behavior, the first judgment result indicates that person area DA2 is an "NG area." In other words, the first judgment result indicates that a situation has occurred in which refueling cannot be permitted in the person area DA2. Here, the inappropriate behavior includes person A being a child, person A smoking a cigarette, person A refueling a refueling target other than vehicle C, and multiple people being present in the person area DA2.

[0078] The refueling non-permission condition for the nozzle area DA3 includes, for example, that the refueling nozzle 41 is not inserted into the fuel filler opening of the vehicle C. When the refueling nozzle 41 is not inserted into the fuel filler opening of the vehicle C, the first judgment result indicates that the nozzle area DA3 is an "NG area." In other words, the first judgment result indicates that a situation has occurred in which refueling action cannot be permitted in the nozzle area DA3. Here, the refueling nozzle 41 not being inserted into the fuel filler opening of the vehicle C includes, in addition to the refueling nozzle 41 not being inserted into the fuel filler opening of the vehicle C, the refueling nozzle 41 being inserted into a refueling target other than the vehicle C, and the like.

[0079] Next, the configuration of the mobile terminal 24 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the configuration of the mobile terminal 24 of this embodiment. As shown in Fig. 6, the mobile terminal 24 includes a terminal communication unit 241, a terminal processing unit 242, a terminal storage unit 243, and a touch display 244.

[0080] The terminal communication unit 241 controls communication between the video processing server 22 and the SSC 6 described with reference to Fig. 1. The terminal communication unit 241 includes, for example, a wireless LAN board or a wireless LAN module.

[0081] The terminal communication unit 241 is an example of a display image receiving unit, a determination result information receiving unit, an operation information transmitting unit, and a request signal transmitting unit. The terminal communication unit 241 receives the display image (captured image T and first presentation image PR1) described with reference to Fig. 1 from the video processing server 22. The terminal communication unit 241 receives the determination result information described with reference to Fig. 4 from the video processing server 22. The terminal communication unit 241 transmits the operation information described with reference to Fig. 1 to the video processing server 22. The terminal communication unit 241 transmits the request signal described with reference to Fig. 1 to the video processing server 22.

[0082] The terminal storage unit 243 includes a semiconductor memory such as a ROM, a RAM, and a flash memory. The terminal storage unit 243 stores various computer programs executed by the terminal processing unit 242. The terminal storage unit 243 also stores various types of data.

[0083] The touch display 244 includes a display device 2441 and a touch panel 2442. The display device 2441 is an example of a display unit. The touch panel 2442 is an example of a first operation receiving unit and a second operation receiving unit.

[0084] The display device 2441 displays the display images (captured image T and first presented image PR1) described with reference to Fig. 1. The display device 2441 is, for example, a liquid crystal display or an organic EL (electroluminescence) display. The touch panel 2442 is superimposed on the display surface of the display device 2441. The touch panel 2442 accepts touch operations by an employee (operator). The touch panel 2442 accepts the refueling permission operation described with reference to Fig. 1.

[0085] The device processing unit 242 executes various processes. The device processing unit 242 has a processor such as a CPU or an MPU. The processor executes computer programs stored in the device storage unit 243 to execute various processes.

[0086] For example, the terminal processing unit 242 controls the terminal communication unit 241 to transmit to the video processing server 22 the request signal (a signal requesting the video processing server 22 to send a display image) described with reference to FIG.

[0087] Furthermore, the device processing unit 242 executes a decision process for deciding whether or not to enable a refueling permission operation for permitting a refueling action by a customer (refueling worker). In detail, the device processing unit 242 decides whether or not to enable the refueling permission operation based on the determination result information described with reference to Fig. 4. The device processing unit 242 is an example of a decision unit.

[0088] In this embodiment, the device processing unit 242 further generates a second presentation image PR2 based on the judgment result information. The second presentation image PR2 indicates the judgment result for each confirmation-required area. Each judgment result indicates whether or not a situation has occurred in which refueling action is not permitted in the corresponding confirmation-required area. In other words, the second presentation image PR2 indicates whether or not an NG area is included in the confirmation-required area. The second presentation image PR2 includes, for example, a character image or a mark image indicating the judgment result.

[0089] In this embodiment, the second presentation image PR2 includes a text image. Hereinafter, the text image indicating that the confirmation-required area is an NG image may be referred to as a "first text image." Similarly, the text image indicating that the confirmation-required area is an OK area may be referred to as a "second text image."

[0090] Next, the monitoring system 2 of this embodiment will be described with reference to Figs. 1 to 9. Figs. 7 to 9 are diagrams showing screens displayed by the mobile terminal 24 of this embodiment. In detail, Fig. 7 shows the mobile terminal 24 displaying the first screen G1. The first screen G1 is a refueling permission issuance screen. Hereinafter, the first screen G1 may be referred to as the "refueling permission issuance screen G1."

[0091] 7, the device processing unit 242 displays a refueling permission issuance screen G1 on the touch display 244. The refueling permission issuance screen G1 displays lane identification information LC, a captured image T, a first presentation image PR1, a second presentation image PR2, and a first button image B1.

[0092] 1, the device processing unit 242 controls the device communication unit 241 to transmit a request signal (a signal requesting a display image) to the video processing server 22. As a result, the device processing unit 242 acquires the determination result information together with the display image (the captured image T and the first presentation image PR1) via the device communication unit 241. The device processing unit 242 generates a second presentation image PR2 based on the determination result information, and causes the touch display 244 to display the second presentation image PR2 together with the display image (the captured image T and the first presentation image PR1).

[0093] The lane identification information LC is information for identifying the target lane LT (the refueling lane L to be monitored). In this embodiment, the lane identification information LC includes a text image indicating the target lane LT. In detail, as described with reference to FIG. 4, the request signal indicates the identification information of the target lane LT. The processing unit 222 of the video processing server 22 transmits the lane identification information LC (text image) to the mobile terminal 24 together with the display image and the judgment result information based on the identification information of the target lane LT.

[0094] The first button image B1 is a button image for permitting the action of refueling. When an employee touches the first button image B1, the device processing unit 242 controls the terminal communication unit 241 to transmit a refueling permission signal to the SSC 6. The touch operation is, for example, a single tap. The operation of touching the first button image B1 is an example of a refueling permission operation. Hereinafter, the first button image B1 may be referred to as the "refueling permission button B1."

[0095] 7, the first presentation image PR1 includes a text image "CAR AREA" indicating the vehicle area DA1, a text image "PERSON AREA" indicating the person area DA2, and a text image "NOZZLE" indicating the nozzle area DA3. The second presentation image PR2 includes a text image "NG" as the first text image (text image indicating that the confirmation-required area is an NG image) and a text image "OK" as the second text image (text image indicating that the confirmation-required area is an OK image).

[0096] 7, the judgment results (first judgment results) of the vehicle area DA1 and the person area DA2 are "OK", that is, refueling is permitted. On the other hand, the first judgment result of the nozzle area DA3 is "NG", that is, refueling is not permitted.

[0097] When the determination result information indicates that all of the check-required areas DA1 to DA3 are OK areas, the device processing unit 242 decides to validate the refueling permission operation. On the other hand, when the determination result information indicates that any of the check-required areas DA1 to DA3 are NG areas, the device processing unit 242 decides not to validate the refueling permission operation. In other words, the device processing unit 242 decides to invalidate the refueling permission operation.

[0098] 7, since the determination result (first determination result) of the nozzle area DA3 is "NG," the device processing unit 242 decides to invalidate the refueling permission operation. In this embodiment, when the device processing unit 242 decides to invalidate the refueling permission operation, it grays out the refueling permission button B1 as shown in FIG. 7, and invalidates the touch operation on the refueling permission button B1.

[0099] Fig. 8 shows the mobile terminal 24 displaying a refueling permission issuance screen G1. More specifically, the refueling permission issuance screen G1 in Fig. 8 displays the captured image T acquired after the captured image T shown in Fig. 7 was captured.

[0100] As shown in Fig. 8, when the determination result information indicates that all of the check-required areas DA1 to DA3 are OK areas, the device processing unit 242 determines to validate the refueling permission operation. Specifically, as shown in Fig. 8, the device processing unit 242 validates the touch operation on the refueling permission button B1.

[0101] Fig. 9 shows an example of a refueling permission issuance screen G1 displayed on the mobile terminal 24 when the first determination result by the video processing server 22 includes an erroneous determination. In the example shown in Fig. 9, the determination results (first determination result) of the vehicle area DA1 and the person area DA2 are "OK", and the determination result (first determination result) of the nozzle area DA3 is "NG". In this case, the employee checks the nozzle area DA3 via the refueling permission issuance screen G1, and if there is no factor in the nozzle area DA3 that would disallow the refueling action, performs a touch operation on the nozzle area DA3.

[0102] When an employee performs a touch operation, the device processing unit 242 controls the terminal communication unit 241 to transmit, to the video processing server 22, coordinate information (touch coordinates) indicating the position where the touch operation was performed in the captured image T and touch operation information indicating the type of the touch operation, based on a signal output from the touch panel 2442. The coordinate information and the touch operation information are examples of operation information (information indicating the content of the operation performed on the displayed image).

[0103] When the processing unit 222 of the video processing server 22 acquires the coordinate information and the touch operation information via the communication unit 221, the processing unit 222 determines whether or not a predetermined operation has been performed on the confirmation-required areas DA1 to DA3 in the captured image T based on the coordinate information and the touch operation information (second determination process). The predetermined operation is, for example, a single tap.

[0104] The processing unit 222 of the video processing server 22 executes a fixing process based on the second determination result (the determination result of the second determination process). Specifically, when the processing unit 222 of the video processing server 22 determines that a predetermined operation has been performed on the check-required areas DA1 to DA3 in the captured image T, it fixes the determination result of the check-required area where the predetermined operation has been performed to "OK". Then, it controls the communication unit 221 to transmit the determination result information after the fixing process to the mobile terminal 24.

[0105] For example, if the employee single-tap on the nozzle area DA3, the second judgment result indicates that the nozzle area DA3 has been confirmed. Therefore, the processing unit 222 of the video processing server 22 fixes the judgment result for the nozzle area DA3 to "OK." Therefore, the judgment result information indicates that the nozzle area DA3 is an OK area.

[0106] When the device processing unit 242 receives the determination result information after the fixing process via the device communication unit 241, it updates the second presentation image PR2 based on the determination result information after the fixing process. For example, if the nozzle area DA3 is an OK area in the determination result information after the fixing process, the device processing unit 242 changes (fixes) the character image for the nozzle area DA3 in the second presentation image PR2 from the first character image (NG) to the second character image (OK).

[0107] When the device processing unit 242 receives the determination result information after the fixing process via the device communication unit 241, it determines whether or not to enable the refueling permission operation based on the determination result information after the fixing process. For example, in the example shown in Fig. 9, when the determination result for the nozzle area DA3 is fixed to "OK", all of the confirmation required areas DA1 to DA3 become OK areas, so it determines to enable the refueling permission operation. As a result, as shown in Fig. 8, the touch operation on the refueling permission button B1 is enabled.

[0108] Next, the process executed by the processing unit 222 of the video processing server 22 will be described with reference to Fig. 1 to Fig. 10. Fig. 10 is a flowchart showing the process executed by the processing unit 222 of the video processing server 22 of this embodiment. The process shown in Fig. 10 is started when the mobile terminal 24 transmits a request signal to the video processing server 22. Note that the video processing server 22 constantly acquires the first captured image T1 and the second captured image T2 from the first imaging device 26a and the second imaging device 26b, and constantly extracts a check-required area in the first captured image T1 and a check-required area in the second captured image T2.

[0109] 10, when the processing unit 222 of the video processing server 22 receives a request signal from the mobile terminal 24, it executes a first determination process for a check-required area in the captured image T of the target lane LT (step S1). As described with reference to FIG. 4, the first determination process is a process for determining whether or not the image of the check-required area is an image that matches the refueling non-permission condition.

[0110] The processor 222 of the video processing server 22 executes the first determination process to generate determination result information. The determination result information indicates whether the determination result for each check-required area is "NG" or not.

[0111] After executing the first determination process, the processing unit 222 of the video processing server 22 controls the communication unit 221 to transmit the determination result information together with the captured image T of the target lane LT and the first presented image PR1 (display image) to the mobile terminal 24 (step S2).

[0112] After transmitting the display image and the determination result information, the processing unit 222 of the video processing server 22 determines whether or not the check-required region includes an NG region based on the determination result information (step S3).

[0113] When the processor 222 of the video processing server 22 determines that the check-to-be-checked regions do not include any NG regions (No in step S3), the process shown in FIG. 10 ends.

[0114] When the processor 222 of the video processing server 22 determines that the check-to-be-performed area is included in the NG area (Yes in step S3), the processor 222 determines whether or not operation information (coordinate information and touch operation information) has been received from the mobile terminal 24 (step S4).

[0115] If the processing unit 222 of the video processing server 22 determines that no operation information has been received from the mobile terminal 24 (No in step S4), the process returns to step S1.

[0116] When it is determined that operation information has been received from the mobile terminal 24 (Yes in step S4), the processing unit 222 of the video processing server 22 determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image T based on the operation information (step S5). That is, the processing unit 222 of the video processing server 22 executes a second determination process.

[0117] When the processing unit 222 of the video processing server 22 determines that a predetermined operation has not been performed on the check-required area in the captured image T (No in step S5), the process returns to step S1.

[0118] When the processor 222 of the video processing server 22 determines that a predetermined operation has been performed on the check-required area in the captured image T (Yes in step S5), it executes a fixing process (step S6). In detail, the processor 222 of the video processing server 22 fixes the determination result for the check-required area where the predetermined operation has been performed to "OK" in the determination result information.

[0119] After the fixing process is performed, the process returns to the process of step S1. As already described, the video processing server 22 acquires the first captured image T1 and the second captured image T2 from the first imaging device 26a and the second imaging device 26b at any time, and extracts the confirmation-required area in the first captured image T1 and the confirmation-required area in the second captured image T2 at any time. Therefore, when returning to the process of step S1 from the process of step S4, step S5, or step S6, the determination result of the first determination process may change. Note that, since the determination result of the confirmation-required area in which a predetermined operation has been performed is fixed to "OK" by the fixing process, the processing unit 222 of the video processing server 22 performs the first determination process on the confirmation-required area in which the determination result is not fixed to "OK" among the confirmation-required areas.

[0120] Next, the process executed by the device processing unit 242 will be described with reference to Fig. 1 to Fig. 11. Fig. 11 is a flowchart showing the process executed by the device processing unit 242 of the mobile terminal 24 of this embodiment. The process shown in Fig. 11 is started when the mobile terminal 24 transmits a request signal to the video processing server 22.

[0121] As shown in FIG. 11, the terminal processing unit 242 transmits a request signal to the image processing server 22, and then receives the determination result information and the display image of the target lane LT (the captured image T and the first presentation image PR1) from the image processing server 22 (step S11).

[0122] When the device processing unit 242 receives the determination result information and the display image of the target lane LT from the video processing server 22, it generates the second presentation image PR2 described with reference to Fig. 6 based on the determination result information (step S12). Then, the device processing unit 242 causes the touch display 244 to display the second presentation image PR2 together with the display image (the captured image T and the first presentation image PR1).

[0123] After displaying the display image (captured image T and first presentation image PR1) and the second presentation image PR2 on the touch display 244, the device processing unit 242 determines whether or not to enable the refueling permission operation based on the judgment result information (step S13). More specifically, the device processing unit 242 determines whether or not the confirmation-required area includes an NG area based on the judgment result information.

[0124] When the terminal processing unit 242 determines that the NG area is included in the confirmation-required area, it determines not to validate the refueling permission operation (No in step S13). In other words, the terminal processing unit 242 determines to invalidate the refueling permission operation.

[0125] After determining that the refueling permission operation is not to be enabled, the device processing unit 242 determines whether or not a touch operation has been performed based on a signal output from the touch display 244 (touch panel 2442) (step S14).

[0126] If the device processing unit 242 determines that a touch operation has not been performed (No in step S14), the process returns to step S11.

[0127] When it is determined that a touch operation has been performed (Yes in step S14), the device processing unit 242 controls the terminal communication unit 241 to transmit operation information (coordinate information and touch operation information) to the video processing server 22 (step S15). As a result, the process returns to the process in step S11.

[0128] When the device processing unit 242 determines that the check-required area does not include a NG area, it determines to enable the refueling permission operation (Yes in step S13). After determining to enable the refueling permission operation, the device processing unit 242 determines whether or not the refueling permission button B1 described with reference to FIG. 7 has been touched (step S16).

[0129] The device processing unit 242 waits until the refueling permission button B1 is touched (No in step S16). When the device processing unit 242 determines that the refueling permission button B1 is touched (Yes in step S16), it controls the terminal communication unit 241 to transmit a refueling permission signal to the SSC 6 (step S17), and ends the process shown in FIG.

[0130] 10, the determination result information generated by the image processing server 22 and the captured image T acquired by the image processing server 22 from the imaging device 26 are updated as needed. Therefore, the determination result information and the display image of the target lane LT received by the device processing unit 242 in step S11 are updated as needed.

[0131] The first embodiment of the present invention has been described above with reference to Fig. 1 to Fig. 11. According to this embodiment, the mobile terminal 24 displays the first presentation image PR1 (an image presenting a confirmation-required area in the captured image T), so that a customer at a refueling station can more reliably confirm the safety of the refueling action performed by the customer himself / herself.

[0132] Furthermore, according to this embodiment, the mobile terminal 24 displays the second presentation image PR2 (an image indicating whether the confirmation area is an NG area or not), thereby enabling customers at a refueling station to more reliably confirm the safety of their own refueling actions.

[0133] Furthermore, according to this embodiment, the refueling permission operation is not valid until an employee checks the confirmation-required area that has been determined to be an NG area by the image processing server 22. Therefore, the safety of the refueling action by the customer at the refueling station can be more reliably confirmed.

[0134] In this embodiment, the device processing unit 242 generates the second presentation image PR2, but the processing unit 222 of the video processing server 22 may generate the second presentation image PR2.

[0135] In addition, in this embodiment, the video processing server 22 transmitted information indicating the judgment result for each area requiring confirmation as judgment result information to the mobile terminal 24, but the video processing server 22 may also transmit the second presentation image PR2 as the judgment result information to the mobile terminal 24.

[0136] In this case, the processing unit 222 of the video processing server 22 generates a second presentation image PR2 based on the first determination result and the second determination result. Moreover, the device processing unit 242 does not generate the second presentation image PR2, but acquires a determination result for each confirmation-required region based on the second presentation image PR2 received from the video processing server 22. For example, the device processing unit 242 may acquire the determination result by character recognition processing. The character recognition processing is, for example, optical character recognition (OCR) processing.

[0137] [Embodiment 2] Next, a second embodiment of the present invention will be described with reference to Figures 1 to 6 and 12 to 14. However, differences from the first embodiment will be described, and a description of the same matters as in the first embodiment will be omitted. In the second embodiment, unlike the first embodiment, the video processing server 22 does not execute the first determination process.

[0138] Fig. 12 is a diagram showing a screen displayed by the mobile terminal 24 of this embodiment. In detail, Fig. 12 shows the mobile terminal 24 displaying the refueling permission issuance screen G1. As shown in Fig. 12, the mobile terminal 24 of this embodiment does not display the second presentation image PR2 described with reference to Fig. 6. In this embodiment, the refueling permission operation is enabled by performing a predetermined operation on the entire area of ​​the confirmation required area.

[0139] Next, the process executed by the processing unit 222 of the video processing server 22 will be described with reference to Figs. 1 to 6, 12, and 13. Fig. 13 is a flowchart showing the process executed by the processing unit 222 of the video processing server 22 of this embodiment. The process shown in Fig. 13 is started when the mobile terminal 24 transmits a request signal to the video processing server 22. Note that, similarly to the first embodiment, the video processing server 22 constantly acquires the first captured image T1 and the second captured image T2 from the first imaging device 26a and the second imaging device 26b, and constantly extracts a check-required area in the first captured image T1 and a check-required area in the second captured image T2.

[0140] As shown in FIG. 13, when the processing unit 222 of the video processing server 22 receives a request signal from the mobile terminal 24, it controls the communication unit 221 to transmit a display image (the first presented image PR1 and the captured image T of the target lane LT) to the mobile terminal 24 (step S21).

[0141] After transmitting the display images (the first presentation image PR1 and the captured image T of the target lane LT), the processing unit 222 of the video processing server 22 determines whether or not operation information (coordinate information and touch operation information) has been received from the mobile terminal 24 (step S22).

[0142] If the processing unit 222 of the video processing server 22 determines that no operation information has been received from the mobile terminal 24 (No in step S22), the process returns to step S21.

[0143] When it is determined that operation information has been received from the mobile terminal 24 (Yes in step S22), the processing unit 222 of the video processing server 22 determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image T based on the operation information (step S23). That is, the processing unit 222 of the video processing server 22 executes a second determination process.

[0144] If the processing unit 222 of the video processing server 22 determines that a predetermined operation has not been performed on the check-required area in the captured image T (No in step S23), the process returns to step S21.

[0145] When the processing unit 222 of the video processing server 22 determines that a predetermined operation has been performed on the check-required area in the captured image T (Yes in step S23), it controls the communication unit 221 to transmit the determination result information to the mobile terminal 24 (step S24). Here, the determination result information indicates the second determination result. In other words, the determination result information is information indicating whether or not the check-required area has been confirmed.

[0146] After transmitting the determination result information, the processor 222 of the video processing server 22 determines whether or not the entire area of ​​the confirmation-required area has been confirmed based on the determination result information (step S25). In other words, the processor 222 of the video processing server 22 determines whether or not a predetermined operation has been performed on the entire area of ​​the confirmation-required area based on the determination result information.

[0147] If the processor 222 of the video processing server 22 determines that any of the confirmation-required areas has not been confirmed (No in step S25), the process returns to step S21.

[0148] When the processor 222 of the video processing server 22 determines that the entire confirmation-required region has been confirmed (Yes in step S25), the processor 222 ends the process shown in FIG.

[0149] Next, the process executed by the device processing unit 242 of the mobile terminal 24 will be described with reference to Fig. 1 to Fig. 6 and Fig. 12 to Fig. 14. Fig. 14 is a flowchart showing the process executed by the device processing unit 242 of the mobile terminal 24 of this embodiment. The process shown in Fig. 14 is started when the mobile terminal 24 transmits a request signal to the video processing server 22.

[0150] 14, the device processing unit 242 transmits a request signal to the video processing server 22, and then receives a display image of the target lane LT (the captured image T and the first presentation image PR1) from the video processing server 22 (step S31). Upon receiving the display image of the target lane LT from the video processing server 22, the device processing unit 242 causes the touch display 244 to display the display image.

[0151] After causing the touch display 244 to display the display image, the device processing unit 242 determines whether or not a touch operation has been performed based on a signal output from the touch display 244 (touch panel 2442) (step S32).

[0152] If the device processing unit 242 determines that a touch operation has not been performed (No in step S32), the process returns to the process in step S31.

[0153] When it is determined that a touch operation has been performed (Yes in step S32), the device processing unit 242 controls the terminal communication unit 241 to transmit operation information (coordinate information and touch operation information) to the video processing server 22 (step S33).

[0154] After transmitting the operation information, the device processing unit 242 determines whether or not the determination result information has been received from the video processing server 22 (step S34). Here, the determination result information indicates the second determination result. In other words, the determination result information is information indicating whether or not the confirmation-required area has been confirmed.

[0155] If the terminal processing unit 242 determines that the determination result information has not been received from the video processing server 22 (No in step S34), the process returns to step S31.

[0156] When the device processing unit 242 determines that it has received the determination result information from the video processing server 22 (Yes in step S34), it determines whether or not to enable the refueling permission operation based on the determination result information (step S35). More specifically, the device processing unit 242 determines whether or not the entire area of ​​the confirmation required area has been confirmed based on the determination result information.

[0157] When the terminal processing unit 242 determines that a part of the confirmation-required area has not been confirmed, it decides not to validate the refueling permission operation (No in step S35). In other words, the terminal processing unit 242 decides to invalidate the refueling permission operation. As a result, the process returns to the process in step S31.

[0158] When it is determined that the entire confirmation-required region has been confirmed, the device processing unit 242 determines to enable the refueling permission operation (Yes in step S35). After determining to enable the refueling permission operation, the device processing unit 242 determines whether or not the refueling permission button B1 described with reference to FIG. 7 has been touched (step S36).

[0159] The device processing unit 242 waits until the refueling permission button B1 is touched (No in step S36). When the device processing unit 242 determines that the refueling permission button B1 is touched (Yes in step S36), it controls the terminal communication unit 241 to transmit a refueling permission signal to the SSC 6 (step S37), and ends the process shown in FIG.

[0160] The second embodiment of the present invention has been described above with reference to Figures 1 to 6 and 12 to 14. According to this embodiment, similar to the first embodiment, the mobile terminal 24 displays the first presentation image PR1 (an image presenting a confirmation-required area in the captured image T), so that a customer at a refueling station can more reliably confirm the safety of the refueling action he or she is taking.

[0161] In addition, according to this embodiment, the refueling permission operation is not valid until the entire confirmation area is confirmed by the staff. Therefore, the safety of the refueling action by the customer at the refueling station can be confirmed more reliably.

[0162] In this embodiment, the mobile terminal 24 does not display the second presentation image PR2, but the mobile terminal 24 may display the second presentation image PR2 indicating the second determination result. In this case, the second presentation image PR2 includes only the second character image "OK" out of the first character image "NG" and the second character image "OK". In other words, the second presentation image PR2 presents information indicating the confirmation-required area where a predetermined touch operation has been performed.

[0163] The embodiments of the present invention have been described above with reference to the drawings (FIGS. 1 to 14). However, the present invention is not limited to the above embodiments, and can be implemented in various aspects without departing from the gist of the present invention. In addition, the components disclosed in the above embodiments can be modified as appropriate. For example, a certain component among all the components shown in one embodiment may be added to a component of another embodiment, or some components among all the components shown in one embodiment may be deleted from the embodiment.

[0164] The drawings are mainly schematic illustrations of each component to facilitate understanding of the invention, and the thickness, length, number, spacing, etc. of each component shown in the drawings may differ from the actual ones due to the convenience of creating the drawings. Furthermore, the configurations of each component shown in the above embodiment are merely examples and are not particularly limited, and it goes without saying that various modifications are possible within a range that does not substantially deviate from the effects of the present invention.

[0165] For example, in the embodiment described with reference to Figures 1 to 14, the weighing machine 4 transmits a fuel supply permission request signal to the SSC 6, but the weighing machine 4 does not have to transmit a fuel supply permission request signal to the SSC 6. In this case, the SSC 6 transmits a monitoring start signal to the mobile terminal 24 in response to receiving data indicating the type of fuel selected by the customer, the payment method, and the amount of fuel supplied or the payment amount.

[0166] In addition, in the embodiment described with reference to Figures 1 to 14, one imaging device 26 is provided for each refueling lane L or refueling area P, but multiple imaging devices 26 may be provided for each refueling lane L or refueling area P.

[0167] In addition, in the embodiment described with reference to Figures 1 to 14, when the device processing unit 242 decides to disable the refueling permission operation, the first button image B1 (refueling permission button B1) is grayed out, but when the device processing unit 242 decides to disable the refueling permission operation, the first button image B1 (refueling permission button B1) does not have to be displayed.

[0168] 1 to 14, the mobile terminal 24 is the refueling permission issuing terminal, but the refueling permission issuing terminal may be the SSC 6. In this case, the SSC 6 includes at least one of a touch panel and a pointing device.

[0169] Furthermore, the mobile terminal 24 may enlarge and display the area of ​​the captured image T where the touch operation was performed in response to a touch operation on the first screen G1 (fueling permission issuance screen). Hereinafter, the mobile terminal 24 of another embodiment will be described with reference to FIG. 15. FIG. 15 is a diagram showing a screen displayed by the mobile terminal 24 of the other embodiment. In detail, FIG. 15 shows the mobile terminal 24 displaying the second screen G2. The second screen G2 displays lane identification information LC, an enlarged image TA, and a second button image B2.

[0170] When the device processing unit 242 detects, based on a signal output from the touch display 244 (touch panel 2442), that a touch operation has been performed to instruct enlarging a portion of the captured image T, the device processing unit 242 controls the device communication unit 241 to transmit operation information to the video processing server 22. Here, the operation information includes coordinate information and touch operation information. The touch operation information indicates the type of touch operation. For example, the touch operation instructing to enlarge a portion of the captured image T is a single tap. In this case, the touch operation indicating that the employee has confirmed the area to be checked is a touch operation other than a single tap. For example, the touch operation indicating that the employee has confirmed the area to be checked may be a double tap.

[0171] When the touch operation information included in the operation information received from the mobile terminal 24 indicates a touch operation instructing to enlarge a partial area of ​​the captured image T, the processing unit 222 of the video processing server 22 generates an enlarged image TA by enlarging a partial area of ​​the captured image T based on the coordinate information. Then, the processing unit 222 of the video processing server 22 controls the communication unit 221 to transmit the enlarged image TA to the mobile terminal 24. As a result, the device processing unit 242 causes the second screen G2 to be displayed on the touch display 244.

[0172] The second button image B2 is a button image for transitioning the screen displayed on the touch display 244 from the second screen G2 to the first screen G1. When the employee touches the second button image B2, the screen displayed on the touch display 244 transitions from the second screen G2 to the first screen G1 (fueling permission issuance screen). [Industrial Applicability]

[0173] The present invention can be applied to self-service fuel stations and has industrial applicability. [Explanation of symbols]

[0174] 2: Surveillance system 22: Video processing server 24: Mobile devices 26: Imaging device 221: Communications Department 222: Processing section 223: Storage section 241: Terminal communication unit 242: Terminal processing section 243: Terminal memory section 244: Touch display 2441: Display device 2442: Touch panel DA1~DA3: Area to be confirmed PR1: First presentation image PR2: Second presentation image T: Captured image

Claims

1. A monitoring device communicably connected to a refueling permission issuing terminal that accepts an operation for permitting a refueling operation by a refueling worker, an image receiving unit that receives an image from an imaging device that captures an image of the periphery of a metering machine installed at a fuel station; an area extraction unit that extracts from the captured image a plurality of confirmation-required areas that need to be confirmed in order to permit the fuel supply operation by the fuel supply worker; an image generating unit that generates a display image including the captured image and a presentation image presenting the confirmation-required area; a request signal receiving unit that receives a request signal requesting the display image output by the fuel supply permission issuing terminal based on receipt of a monitoring start signal from a console device connected to the meter; A display image transmission unit that transmits the display image to the refueling permission issuing terminal; a judgment result information generating unit that executes a first judgment process for judging, for each of the confirmation required areas, whether or not the image of the confirmation required area is an image that matches a refueling non-permission condition that prohibits the refueling action by the refueling worker based on the captured image, and generates judgment result information that indicates, for each of the confirmation required areas, whether or not the refueling action is permitted; a determination result information transmitting unit that transmits the determination result information to the refueling permission issuing terminal; an operation information receiving unit that receives operation information indicating the content of an operation performed on the display image from the refueling permission issuing terminal; Equipped with the monitoring device is not connected to the console device except via the refueling permission issuing terminal, the display image is an image that displays the plurality of confirmation-required regions in a identifiable manner on one of the captured images, The determination result information generating unit A monitoring device that generates the judgment result information that validates the refueling permission operation by the refueling permission issuing terminal when it determines, based on the operation information, that a specified operation has been performed for all of the NG areas even if the multiple areas requiring confirmation include an NG area in which the first judgment process has determined that the refueling action is not permitted.

2. The judgment result information generating unit, A second determination process is further performed to determine whether or not a predetermined operation has been performed on the confirmation-required region based on the operation information. A monitoring device as described in claim 1, wherein when it is determined that the specified operation has been performed in the confirmation-required area, the judgment result for the confirmation-required area in which the specified operation has been performed is fixed to a judgment result indicating that the refueling action may be permitted.

3. The judgment result information generating unit, The monitoring device of claim 1, further comprising: a determining unit that determines whether a predetermined operation has been performed on the area to be confirmed based on the operation information; and, if it determines that the predetermined operation has been performed on the area to be confirmed, generates the determination result information indicating that confirmation of the area to be confirmed has been performed.

4. The monitoring device according to claim 1 , wherein the display image transmitting unit transmits the display image in response to the request signal being received by the request signal receiving unit.

5. A monitoring device communicably connected to a refueling permission issuing terminal that accepts an operation for permitting a refueling operation by a refueling worker; An imaging device that captures images of the area around the weighing machine installed at the fuel station; Equipped with The monitoring device includes: a captured image receiving unit that receives a captured image from the imaging device; an area extraction unit that extracts from the captured image a plurality of confirmation-required areas that need to be confirmed in order to permit the fuel supply operation by the fuel supply worker; an image generating unit that generates a display image including the captured image and a presentation image presenting the confirmation-required area; a request signal receiving unit that receives a request signal requesting the display image output by the fuel supply permission issuing terminal based on receipt of a monitoring start signal from a console device connected to the meter; A display image transmission unit that transmits the display image to the refueling permission issuing terminal; a judgment result information generating unit that executes a first judgment process for judging, for each of the confirmation required areas, whether or not the image of the confirmation required area is an image that matches a refueling non-permission condition that prohibits the refueling action by the refueling worker based on the captured image, and generates judgment result information that indicates, for each of the confirmation required areas, whether or not the refueling action is permitted; a determination result information transmitting unit that transmits the determination result information to the refueling permission issuing terminal; an operation information receiving unit that receives operation information indicating the content of an operation performed on the display image from the refueling permission issuing terminal; Equipped with the monitoring device is not connected to the console device except via the refueling permission issuing terminal, the display image is an image that displays the plurality of confirmation-required regions in a identifiable manner on one of the captured images, The determination result information generating unit A monitoring system that generates the judgment result information that validates the refueling permission operation by the refueling permission issuing terminal when it is determined based on the operation information that a specified operation has been performed for all of the NG areas, even if the multiple areas requiring confirmation include an NG area in which the first judgment process has determined that the refueling action is not permitted.

6. The monitoring system of claim 5 further comprising the refueling permit issuing terminal.

7. a fuel supply permission issuing terminal that accepts an operation for permitting a fuel supply operation by a fuel supply worker; A monitoring device communicably connected to the refueling permission issuing terminal; Equipped with The monitoring device includes: an image receiving unit that receives an image from an imaging device that captures an image of the periphery of a metering machine installed at a fuel station; an area extraction unit that extracts from the captured image a plurality of confirmation-required areas that need to be confirmed in order to permit the fuel supply operation by the fuel supply worker; an image generating unit that generates a display image including the captured image and a presentation image presenting the confirmation-required area; a request signal receiving unit that receives a request signal requesting the display image output by the fuel supply permission issuing terminal based on receipt of a monitoring start signal from a console device connected to the meter; A display image transmission unit that transmits the display image to the refueling permission issuing terminal; a judgment result information generating unit that executes a first judgment process for judging, for each of the confirmation required areas, whether or not the image of the confirmation required area is an image that matches a refueling non-permission condition that prohibits the refueling action by the refueling worker based on the captured image, and generates judgment result information that indicates, for each of the confirmation required areas, whether or not the refueling action is permitted; a determination result information transmitting unit that transmits the determination result information to the refueling permission issuing terminal; an operation information receiving unit that receives operation information indicating the content of an operation performed on the display image from the refueling permission issuing terminal; Equipped with the monitoring device is not connected to the console device except via the refueling permission issuing terminal, the display image is an image that displays the plurality of confirmation-required regions in a identifiable manner on one of the captured images, The determination result information generating unit A monitoring system that generates the judgment result information that validates the refueling permission operation by the refueling permission issuing terminal when it is determined based on the operation information that a specified operation has been performed for all of the NG areas, even if the multiple areas requiring confirmation include an NG area in which the first judgment process has determined that the refueling action is not permitted.

8. a display image receiving unit that receives from a monitoring device a display image including an image acquired from an imaging device that captures an image of the periphery of a weighing machine installed at a fuel station and a presentation image that identifiably presents on one of the imaged images a plurality of check-required areas that need to be checked in order to permit a fuel supply operator to perform a fuel supply operation; A display unit that displays the display image; a determination result information receiving unit that receives, from the monitoring device, determination result information indicating whether or not the refueling action is permitted for each of the check-required areas; a decision unit that decides whether or not to enable a refueling permission operation for permitting the refueling action based on the determination result information; a first operation receiving unit that receives the refueling permission operation by an operator; a second operation receiving unit that receives an operation performed on the display image; a request signal transmission unit that transmits a request signal to the monitoring device requesting the display image when a monitoring start signal is received from a console device connected to the weighing machine; a terminal communication unit that, when the refueling permission operation is received, transmits a refueling permission signal indicating permission of the refueling action to the console device without going through the monitoring device; Equipped with The determination unit is A refueling permission issuing terminal that enables the refueling permission operation when a specified operation is performed on all of the NG areas based on the operation accepted by the second operation accepting unit, even if the display image indicates that the multiple areas requiring confirmation include an NG area where the monitoring device has determined that the refueling action is not permitted.

9. Further comprising an operation information transmission unit that transmits operation information indicating the content of an operation performed on the display image to the monitoring device, The monitoring device determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image based on the operation information, The refueling permission issuing terminal of claim 8, wherein when the monitoring device determines that the specified operation has been performed in the area requiring confirmation, the judgment result information receiving unit receives the judgment result information in which the judgment result for the area requiring confirmation is fixed to a judgment result indicating that the refueling action may be permitted.

10. Further comprising an operation information transmission unit that transmits operation information indicating the content of an operation performed on the display image to the monitoring device, the determination result information receiving unit receives, when a predetermined operation is performed on the confirmation-required area, the determination result information indicating that the confirmation-required area has been confirmed from the monitoring device; The monitoring device determines whether or not the predetermined operation has been performed on the confirmation-required area in the captured image based on the operation information; The refueling permission issuing terminal according to claim 8 , wherein the decision unit decides whether or not to validate the refueling permission operation based on the judgment result information.

Images