Oil supply monitoring system

Abstract

To provide an oil supply monitoring system capable of quantifying whether or not a desirable oil supply action has been taken by each oil supplier.SOLUTION: An oil supply monitoring system 100 comprises: an image analysis device 111 that carries out oil supply propriety determination based on a video of a monitoring camera 140 installed in an oil supply station; an evaluation part 201 that calculates an evaluation value related to the action of an oil supplier based on the video of the monitoring camera 140; and a storage part 202 that stores the evaluation value calculated by the evaluation part 201 in association with the oil supplier. The evaluation part 201 performs processing for detecting a predetermined object from the camera video as a determination material for the oil supply propriety determination, and calculates the evaluation value in accordance with a detection result of the object. The evaluation part 201 further performs processing for detecting a correct oil supply action or an incorrect oil supply action based on a sensing result or the camera video, and calculates the evaluation value in accordance with detection results of the actions.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a refueling monitoring system that determines whether or not refueling is possible based on images captured by cameras installed at gas stations. [Background technology]

[0002] In conventional gas stations, gas station staff (supervisors) would monitor the behavior of customers (fuelers) filling up their tanks visually or on a surveillance monitor, and then operate a fuel management device such as a handheld terminal to issue permission to fill up the tank. In recent years, the development of systems that use AI (Artificial Intelligence) to analyze images from surveillance cameras has progressed, and these systems are beginning to be applied to the monitoring of self-service gas stations.

[0003] Here, the following are examples of conventional technologies in the technical field of the present invention. For example, Patent Document 1 discloses an invention that uses AI to analyze images of a person filling up a tank to determine the behavior of the person filling up a tank. Patent Document 2 discloses an invention that efficiently collects learning data for re-learning a learning model used to monitor self-service tankers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-091460 [Patent Document 2] Patent No. 7273109 Summary of the Invention [Problem to be solved by the invention]

[0005] A refueler does not necessarily take desirable refueling actions. For example, if a refueler takes actions that hinder the AI ​​refueling judgment, the judgment accuracy will decrease or errors will occur, and an appropriate judgment result will not be obtained. In this case, the supervisor will have to visually check the refueler's condition. In addition, if the refueler does not take the correct refueling behavior, the supervisor will have to stop the self-refueling and take action. However, the conventional system did not have a mechanism to quantify whether the refueler is taking desirable refueling behavior. [Means for solving the problem]

[0006] In order to achieve the above object, a refueling monitoring system according to one aspect of the present invention is configured as follows: That is, a refueling monitoring system that determines whether refueling is possible based on video footage from a camera installed at a gas station includes an evaluation unit that calculates an evaluation value related to the behavior of a person refueling based on the video footage from the camera, and a storage unit that stores the evaluation value calculated by the evaluation unit in association with the person refueling, and is characterized in that the evaluation unit performs a process of detecting a predetermined object from the video footage from the camera as a criterion for determining whether refueling is possible, and calculates the evaluation value according to a detection result of the object.

[0007] Here, the above-mentioned refueling monitoring system may be configured to further include a display unit that displays the evaluation value calculated by the evaluation unit.

[0008] In addition, in the above-mentioned refueling monitoring system, the evaluation unit may be configured to add an evaluation value when the object can be detected from the camera image, and to subtract the evaluation value when the object cannot be detected from the camera image. Effect of the Invention

[0009] According to the present invention, it is possible to provide a refueling monitoring system that can quantify whether each refueler is taking desirable refueling actions. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of the configuration of a refueling monitoring system according to an embodiment of the present invention; [Diagram 2] FIG. 13 is a diagram showing an example of a refueling history database that manages an evaluation value for each refueling action. [Diagram 3] FIG. 13 is a diagram illustrating an example of a customer database that manages evaluation values ​​for each customer. [Figure 4] FIG. 13 is a diagram showing an example of a process flow relating to evaluation of the behavior of a fuel dispenser. [Diagram 5] FIG. 13 is a diagram showing an example of a processing flow for evaluating whether a refueling action is suitable for judging whether refueling is possible. [Figure 6] FIG. 13 is a diagram showing an example of a process flow for evaluating whether a refueling action is correct or not. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration example of a refueling monitoring system 100 according to an embodiment of the present invention. The refueling monitoring system 100 of this example has a function of making an AI refueling judgment based on images from a camera installed at a gas station. The refueling monitoring system 100 includes a monitoring device 110, a terminal device 120, a refueling management device 130, a weighing machine 135, a monitoring camera 140, and a sensor 150.

[0012] The surveillance camera 140 is installed at an arbitrary position in the gas station, captures the self-service refueling situation, and outputs the obtained captured data (camera video) to the monitoring device 110. The surveillance camera 140 is installed for each fueling lane (vehicle stopping area) adjacent to the weighing machine 135 so as to capture images of vehicles and fuelers (users) in the fueling lane (vehicle stopping area) adjacent to the weighing machine 135. The surveillance camera 140 is installed, for example, in such a position and orientation that it overlooks from above an area including the area between the weighing machine 135 and the vehicle.

[0013] The sensor 150 senses sound, heat, odor, light, oil leakage, and the like that are generated due to the actions of the fuel dispenser, and outputs the results to the monitoring device 110. The sensor 150 also includes a sensor that detects the removal of a fuel nozzle from the metering machine 135. The sensor 150 is installed, for example, in the metering machine 135 or in its vicinity (such as around the fueling lane).

[0014] The monitoring device 110 is installed at a location (for example, inside a building of a gas station) away from the weighing machine 135. The monitoring device 110 has an image analysis device 111, a control device 112, a wireless device 113, and a learning model storage unit 114. The learning model storage unit 114 stores a learning model prepared in advance for AI analysis of camera images. The image analysis device 111 executes a refueling feasibility determination process based on the learning model in the learning model storage unit 114. That is, the image analysis device 111 analyzes the camera images using AI to perform a process of determining whether or not refueling is possible for the vehicle in the camera images (that is, whether or not refueling is permitted). In the refueling feasibility determination process, the sensing result by the sensor 150 is also taken into consideration. The wireless device 113 performs wireless communication with the terminal device 120. The control device 112 is interposed between the monitoring camera 140, the sensor 150, the fuel supply management device 130, the image analysis device 111, and the wireless device 113, and controls the overall operation of these devices / equipment. The monitoring device 110 is, for example, a computer equipped with hardware resources such as a processor and a memory, and is configured so that the processor executes a program for realizing each function according to the present invention. Note that the above-mentioned monitoring device 110 is composed of the image analysis device 111, the control device 112, the wireless device 113, and the learning model storage unit 114, but is not limited thereto, and for example, the image analysis device 111 (including the learning model storage unit 114), the control device 112, and the wireless device 113 may be provided as separate devices.

[0015] In the refueling permission determination process, it is confirmed whether there are any problems with multiple check items when refueling, based on the camera image captured by the surveillance camera 140 and the sensing results by the sensor 150. For example, it is confirmed that the refueling nozzle of the metering machine 135 is properly inserted (fully inserted) into the fuel filler opening of the vehicle, that there is no heat source such as a cigarette, that there is no portable can, that multiple people are not refueling, etc. Then, if it is confirmed that there are no problems with all of these check items, it is determined that the state is a refueling permission possible state in which refueling is permitted, and if not, it is determined that the state is a refueling permission prohibited state in which refueling is not permitted.

[0016] The terminal device 120 is a device used by a worker (monitor) at a gas station. The terminal device 120 has a control unit 121, a communication unit 122, a display unit 123, and an operation unit 124. The control unit 121 comprehensively controls the operation of each unit of the terminal device 120. The communication unit 122 performs wireless communication with the monitoring device 110. The display unit 123 displays various information including the result of the refueling availability determination process performed by the monitoring device 110. The operation unit 124 accepts various user operations including permission and stop of self-refueling. The display unit 123 and the operation unit 124 may be integrated into a touch panel. The terminal device 120 is, for example, a computer equipped with hardware resources such as a processor and a memory, and is configured so that the processor executes a program for realizing each function according to the present invention.

[0017] The terminal device 120 receives and displays the results of the refueling feasibility determination process from the monitoring device 110. The terminal device 120 can also display camera footage along with the results of the refueling feasibility determination process. The worker considers whether to permit or prohibit (not permit) refueling after referring to the information displayed on the terminal device 120, and if it is determined that refueling should be permitted, performs an operation corresponding to that on the terminal device 120. Furthermore, if the worker determines that refueling should be stopped (interrupted) during refueling after permitting it, the worker performs an operation corresponding to that on the terminal device 120.

[0018] When the terminal device 120 receives the above-mentioned operation, it transmits a control signal indicating that operation (for example, a refueling permission signal or a refueling stop signal) to the refueling management device 130. The control signal is accompanied by information identifying the target refueling lane (for example, a lane number). The control signal according to the operation of the terminal device 120 may be transmitted to the refueling management device 130 via the monitoring device 110, or may be transmitted directly to the refueling management device 130 without passing through the monitoring device 110. Here, an example will be described in which the terminal device 120 is a portable terminal such as a tablet (i.e., a handy terminal) that can be carried by an operator, but the terminal device 120 may also be a stationary terminal installed (fixed) at a predetermined position.

[0019] The refueling management device 130 is also referred to as a self-service controller (SSC), and is installed, for example, at a location away from the metering machine 135 (for example, inside a building of a gas station). The refueling management device 130 controls the operation of the metering machine 135 based on a control signal transmitted in response to an operation of the terminal device 120. For example, when the terminal device 120 receives an operation to permit refueling, a refueling permission signal is transmitted from the terminal device 120 to the refueling management device 130. The refueling management device 130 that receives the refueling permission signal drives a pump inside the metering machine 135 corresponding to the refueling permission signal, and thus the metering machine 135 becomes ready to refuel. Also, for example, when the terminal device 120 receives an operation to stop refueling, a refueling stop signal is transmitted from the terminal device 120 to the metering machine 135. The refueling management device 130 that receives the refueling stop signal stops the pump inside the metering machine 135 that corresponds to the refueling stop signal, and as a result, the metering machine 135 is placed in a state in which it is unable to perform refueling.

[0020] The main feature of this system is that it is equipped with a mechanism that can quantify whether each fuel supplier is taking the desired fueling action. The mechanism is explained in detail below.

[0021] In this system, the control device 112 in the monitoring device 110 has an evaluation unit 201 and a storage unit 202, as shown in Fig. 1. The evaluation unit 201 calculates an evaluation value for the behavior of the refueling person based on the video of the monitoring camera 140. The storage unit 202 stores the evaluation value calculated by the evaluation unit 201 in association with the refueling person. The evaluation value is a numerical representation of the result of evaluating the behavior of the refueling person from the viewpoint of whether it is a desirable refueling behavior or not.

[0022] Here, desirable refueling behavior includes not only correct behavior (appropriate behavior) when refueling, but also behavior for which AI can easily determine whether refueling is permitted. In other words, even if a behavior is incorrect (inappropriate behavior) when refueling, if AI can accurately determine that refueling is not permitted, it can be said to be desirable from the perspective of the entire system. Therefore, the evaluation unit 201 in the refueling monitoring system 100 of this example calculates an evaluation value regarding the behavior of the refueler from two perspectives: (1) whether the refueling behavior is easy for AI to determine whether refueling is permitted, and (2) whether the refueling behavior is correct.

[0023] (1) Refueling behaviors that can be easily determined using AI When AI is used to determine whether refueling is possible, it determines whether the correct refueling action is being taken by analyzing camera footage of the refueler, the refueling nozzle, and other objects. Therefore, to accurately determine whether refueling is possible, it is necessary to obtain camera footage that properly captures the objects detected as information for determining refueling actions. On the other hand, camera footage that does not allow the appearance of the objects (for example, camera footage that does not capture the objects) is not suitable for AI to determine whether refueling is possible.

[0024] (2) Proper refueling procedures When a fuel dispenser performs a correct fuel dispense action, it can be said that this was a desirable fuel dispense action. Conversely, when a fuel dispenser performs an incorrect fuel dispense action, it can be said that this was an undesirable fuel dispense action. A correct fuel dispense action can be determined, for example, based on whether or not the sensor 150 detects contact with an antistatic sheet and correct insertion of a nozzle. An incorrect fuel dispense action can be determined, for example, based on whether or not the sensor 150 detects a heat source such as a cigarette. In addition, the implementation status of a correct or incorrect fuel dispense action may be determined from the result of the AI's determination of whether or not fuel dispense is possible, or may be determined from the result of analyzing camera footage using a method other than the AI's determination of whether or not fuel dispense is possible.

[0025] Based on the above considerations, the evaluation unit 201 of the present system performs processing to detect a predetermined object from the camera image, which is used as a basis for determining whether or not refueling is possible, from the viewpoint (1) above, and evaluates the behavior of the refueler according to the detection result of the object. Furthermore, the evaluation unit 201 of the present system further performs processing to detect correct or incorrect refueling behavior based on the sensing result or the camera image, from the viewpoint (2) above, and evaluates the behavior of the refueler according to the detection result of the behavior. Note that it is desirable to perform both of these processes, but it is also possible to perform only one of these processes.

[0026] The evaluation value calculated as the evaluation result by the evaluation unit 201 is stored in the storage unit 202 in association with the target refueling person. Fig. 2 is an example of a refueling history database that manages an evaluation value for each refueling action. Fig. 3 is an example of a customer database that manages an evaluation value for each customer (refueler).

[0027] As shown in FIG. 2, the refueling history database stores refueling history data including the "refueling date and time" which is the date and time when refueling was performed, the "customer number" which identifies the refueler, the "judgment result" which is the result of the judgment on whether refueling was possible, and the "evaluation value" which is the evaluation result of the refueling action. As shown in FIG. 3, the customer database stores customer data including the "customer number" which identifies the refueler, and the "evaluation value" which is the evaluation result of the refueling action. The evaluation value in the customer database is the cumulative evaluation value of the target refueler up to now. In this example, the evaluation value is stored in association with the customer number which identifies the refueler, but the evaluation value may also be stored in association with other information such as the vehicle number which identifies the vehicle to be refueled.

[0028] Also, the evaluation value calculated by the evaluation unit 201 may be transmitted to the terminal device 120 and displayed thereon. In this way, the worker using the terminal device 120 can recognize whether the refueling person has performed desirable refueling behavior by looking at the evaluation value displayed on the terminal device 120. If the refueling behavior is undesirable, it is possible to encourage the refueling person to perform desirable refueling behavior by requesting the refueling person to improve his / her behavior. Also, the evaluation value calculated by the evaluation unit 201 may be transmitted to a monitor (not shown) attached to the weighing machine 135 used by the refueling person so that the refueling person can recognize the evaluation value.

[0029] Furthermore, before a fuel dispenser starts a fuel dispenser action, the past evaluation value of the fuel dispenser (e.g., evaluation value history or cumulative value) may be read from the customer database and transmitted to and displayed on the terminal device 120. This allows a worker using the terminal device 120 to prompt a fuel dispenser with a low evaluation value (i.e., a fuel dispenser who has previously engaged in undesirable fuel dispenser actions) to engage in desirable fuel dispenser actions in advance.

[0030] FIG. 4 shows an example of a process flow relating to the evaluation process of the behavior of a fuel supplier. In the control device 112 in the monitoring device 110, the evaluation unit 201 first acquires the customer number of the person who is going to refuel (step S101). The customer number can be acquired by any method. For example, the customer number can be acquired through a membership card or a membership application installed on a mobile terminal such as a smartphone. Alternatively, the vehicle and the person who is going to refuel may be identified based on the results of a vehicle number detection process for the license plate portion of the vehicle in the camera image, or a face recognition process for the face portion of the person who is going to refuel in the camera image, and the customer number of the person who is going to refuel may be acquired.

[0031] Thereafter, during the refueling period from the start to the end of a series of procedures related to refueling (steps S102 to S103), the evaluation unit 201 performs a process of detecting a predetermined object from the video of the surveillance camera 140, and records the data of the detection results. During the same period, the evaluation unit 201 further performs a process of detecting correct or incorrect refueling behavior based on the sensing result of the sensor 150 and the video of the surveillance camera 140, and records the data of the detection results. For example, the evaluation unit 201 records the data of each detection result by appropriately switching on / off a flag prepared in advance for each object or behavior to be detected.

[0032] After the fuel supply is completed, the evaluation unit 201 calculates an evaluation value that evaluates the behavior of the fuel supplier based on the detection result data recorded during the above period (steps S102 to S103) (step S104). The process of calculating the evaluation value will be described with reference to Figs. 5 and 6.

[0033] 5 shows an example of a process flow for evaluating whether a refueling action is suitable for AI-based refueling judgment. The evaluation unit 201 refers to data resulting from detection of an object from a camera image and judges whether a nozzle has been detected from the camera image (step S121). As a result, if a refueling nozzle can be detected from the camera image, an evaluation value is added (step S122). On the other hand, if a refueling nozzle cannot be detected from the camera image, an evaluation value is subtracted (step S123).

[0034] The evaluation unit 201 further judges whether or not a fuel supplier is detected from the camera image (step S124). As a result, if a fuel supplier is detected from the camera image, an evaluation value is increased (step S125). On the other hand, if a fuel supplier is not detected from the camera image, an evaluation value is decreased (step S126).

[0035] 6 shows an example of a process flow for evaluating whether the refueling action is correct or not. The evaluation unit 201 refers to the data resulting from detection of correct or incorrect refueling action, and judges whether the refueler touched the static electricity removal sheet (step S141). As a result, if the refueler touched the static electricity removal sheet, an increase in the evaluation value is performed (step S142). On the other hand, if the refueler did not touch the static electricity removal sheet, a decrease in the evaluation value is performed (step S143).

[0036] The evaluation unit 201 further judges whether the fuel dispenser has inserted the fuel dispenser nozzle correctly (step S144). If the fuel dispenser has inserted the fuel dispenser nozzle correctly, the evaluation value is increased (step S145). On the other hand, if the fuel dispenser has not inserted the fuel dispenser nozzle correctly, the evaluation value is decreased (step S146).

[0037] The evaluation unit 201 further judges whether the fuel supplier operated the mobile phone (step S147). If the fuel supplier did not operate the mobile phone, the evaluation value is increased (step S148). On the other hand, if the fuel supplier operated the mobile phone, the evaluation value is decreased (step S149).

[0038] 5 and 6, the amount by which the evaluation value is added when the refueling action is desirable may be constant regardless of the refueling action, or may be changed depending on the importance of the refueling action, etc. Similarly, the amount by which the evaluation value is subtracted when the refueling action is undesirable may be constant regardless of the refueling action, or may be changed depending on the importance of the refueling action, etc. Furthermore, it goes without saying that the judgment conditions shown in the processing flows of FIGS. 5 and 6 are merely examples, and are not limited to these.

[0039] After calculating the evaluation value as described above, the evaluation unit 201 generates refueling history data including the calculated evaluation value and stores it in the refueling history database in the storage unit 202 (step S105). The evaluation unit 201 further reads customer data corresponding to the refueler from the customer database in the storage unit 202, updates the customer data by adding or subtracting the calculated evaluation value, and stores it in the customer database in the storage unit 202 (step S106).

[0040] As described above, the refueling monitoring system 100 of this example includes an image analyzer 111 that determines whether or not refueling is possible based on the image of the surveillance camera 140 installed at the refueling station, an evaluation unit 201 that calculates an evaluation value regarding the behavior of the refueler based on the image of the surveillance camera 140, and a storage unit 202 that stores the evaluation value calculated by the evaluation unit 201 in association with the refueler. The evaluation unit 201 then performs a process of detecting a predetermined object from the camera image as a criterion for determining whether or not refueling is possible, and calculates an evaluation value according to the detection result of the object. The evaluation unit 201 further performs a process of detecting correct or incorrect refueling behavior based on the sensing result or the camera image, and calculates an evaluation value according to the detection result of the behavior. With this configuration, the refueling monitoring system 100 of this example can quantify whether or not each refueler is performing a desirable refueling behavior.

[0041] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

[0042] Furthermore, the present invention can be provided not only as devices such as those described above or as a system composed of these devices, but also as methods executed by these devices, programs for causing a processor to realize the functions of these devices, and storage media for storing such programs in a computer-readable format. [Industrial Applicability]

[0043] The present invention can be used in a refueling monitoring system that uses AI to determine whether or not refueling is possible based on images from cameras installed at gas stations. [Explanation of symbols]

[0044] 100: refueling monitoring system, 110: monitoring device, 111: image analysis device, 112: control device, 113: wireless device, 114: learning model storage unit, 120: terminal device, 121: control unit, 122: communication unit, 123: display unit, 124: operation unit, 130: refueling management device, 135: weighing machine, 140: surveillance camera, 150: sensor, 201: evaluation unit, 202: storage unit

Claims

1. In a refueling monitoring system that determines whether or not refueling is possible based on images from a camera installed at a gas station, An evaluation unit that calculates an evaluation value regarding the behavior of the fuel supplier based on the image of the camera; a storage unit that stores the evaluation value calculated by the evaluation unit in association with the fuel supplier; A refueling monitoring system characterized in that the evaluation unit performs a process of detecting specified objects from the camera image as criteria for determining whether or not refueling is possible, and calculates the evaluation value based on the detection results of the objects.

2. 2. The refueling monitoring system according to claim 1, A refueling monitoring system further comprising a display unit that displays the evaluation value calculated by the evaluation unit.

3. 3. The refueling monitoring system according to claim 1, A refueling monitoring system characterized in that the evaluation unit adds an evaluation value when the object can be detected from the camera image, and subtracts the evaluation value when the object cannot be detected from the camera image.

Images