Load factor estimation device

Abstract

The load factor estimation device of the present invention can improve the accuracy of load factor estimation. The load factor estimation device includes: an identification unit that acquires at least one of a second image and a first image captured inside the loading platform, and identifies and tracks people and objects other than cargo in the acquired image to infer whether the people and the objects other than cargo are present on the loading platform; the second image is an image captured of the area outside the loading platform within a predetermined distance from the loading platform's entrance / exit; and an estimation unit that, if the people and the objects other than cargo are not present on the loading platform, infers the load factor of a vehicle equipped with the loading platform.

Description

Technical Field

[0001] This invention relates to a load factor estimation device used in vehicles. Background Technology

[0002] Previously, methods for inferring the load factor of vehicles equipped with cargo platforms (e.g., cargo boxes or flatbed cargo platforms) were known. For example, Patent Document 1 discloses a method for calculating the load factor based on cargo volume data in the cargo box measured by an ultrasonic sensor.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2004-284722 Summary of the Invention

[0006] The problem the invention aims to solve

[0007] However, previous methods have the following problem: when there are people and objects other than cargo on the loading platform, the people and objects are treated the same as the cargo, which leads to a decrease in the accuracy of the load factor inference.

[0008] One objective of the present invention is to provide a load factor estimation device that can improve the accuracy of load factor estimation.

[0009] Solution to the problem

[0010] One aspect of the present invention provides a cargo load factor estimation device comprising: an identification unit that acquires at least one of a second image and a first image captured of the interior of a cargo platform, and identifies and tracks a person and an object other than cargo in the acquired image, thereby estimating whether the person and the object other than cargo are present on the cargo platform; the second image being an image captured of the exterior of the cargo platform within a predetermined distance from the entrance / exit of the cargo platform; and an estimation unit that, if the person and the object other than cargo are not present on the cargo platform, estimates the cargo load factor of a vehicle equipped with the cargo platform.

[0011] Invention Effects

[0012] According to the present invention, the accuracy of cargo load factor estimation can be improved. Attached Figure Description

[0013] Figure 1 This is a side view schematically illustrating an embodiment of the vehicle of the present invention.

[0014] Figure 2 This is a block diagram illustrating an example of the structure of a load factor estimation device according to an embodiment of the present invention.

[0015] Figure 3 This is a flowchart illustrating an example of the operation of the load factor estimation device according to an embodiment of the present invention. Detailed Implementation

[0016] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0017] First, use Figure 1 The vehicle V in this embodiment will be described. Figure 1 This is a schematic side view of vehicle V.

[0018] like Figure 1 As shown, vehicle V is a truck equipped with a cab 1 and a cargo box 2. It should be noted that vehicle V is not limited to trucks and can also be other types of vehicles.

[0019] Cargo compartment 2 (an example of a loading platform) is, for example, box-shaped, with an opening on its rear side serving as the entrance and exit of cargo compartment 2 (illustrations omitted; the same applies below). Goods are loaded and unloaded through this opening.

[0020] A door 3 that can be opened and closed freely is provided at the rear of the cargo compartment 2, corresponding to the position of the opening. The door 3 may be, for example, a door that opens by rotating from the center of the opening to the left or right with the left and right ends of the opening as the axis (a so-called left-right opening door), but is not limited to this.

[0021] An in-cargo camera 4 is installed at the rear of the cargo compartment 2.

[0022] The camera 4 inside the cargo compartment takes pictures of the entire interior of the cargo compartment 2. Hereinafter, the images taken by the camera 4 inside the cargo compartment will be referred to as "cargo compartment images (an example of the first image)".

[0023] The in-cargo camera 4 captures images of the interior of the cargo compartment and transmits them to the cargo load factor estimation device 100 (see reference). Figure 2 )send.

[0024] Additionally, the in-carriage camera 4 is integrated with a depth sensor (not shown). The depth sensor is a sensor capable of measuring the distance from itself to a person or object in two dimensions. The detection result from the depth sensor is output to the cargo load factor estimation device 100 (see reference 100). Figure 2 It should be noted that the in-carriage camera 4 and the depth sensor can also be separate units.

[0025] A rear camera 5 is installed at the rear of the cargo compartment 2.

[0026] The rear camera 5 is a camera that captures images of the exterior and rear of the vehicle V (specifically, near the outer side of the opening). Hereinafter, the image captured by the rear camera 5 will be referred to as the "first image near the opening (an example of the second image)".

[0027] For example, the area captured by the rear camera 5 does not include the interior of the cargo compartment 2, and is limited to the opening (which can be described as...). Figure 1 The image refers to the area within a specified distance (e.g., several meters) from the door 3 shown. That is, the image near the first opening is an image taken of the exterior of the cargo compartment 2 within a specified distance from the opening.

[0028] The rear camera 5 transmits the image captured near the first opening to the cargo rate estimation device 100 (see reference). Figure 2 )send.

[0029] It should be noted that the installation locations of the in-carriage camera 4 and the rear camera 5 are not limited to... Figure 1 The location shown.

[0030] In addition, although Figure 1 The illustration is omitted, but the vehicle V is also equipped with a cargo rate estimation device 100 (described later) (see reference). Figure 2 ).

[0031] exist Figure 1 The image shows vehicle V parked near berth B. Berth B is a space located in a logistics facility (e.g., a warehouse, distribution center, etc.) for loading or unloading operations of cargo compartment 2.

[0032] like Figure 1 As shown, a surveillance camera 6 is installed above berth B (e.g., the roof section).

[0033] Surveillance camera 6 is filming the area near the entrance to berth B. Thus, for example, as... Figure 1 As shown, when vehicle V stops near the entrance of parking space B, the exterior and rear of vehicle V (specifically, near the outside of the opening) are captured by monitoring camera 6. Hereinafter, the image captured by monitoring camera 6 will be referred to as the "second image near the opening (an example of the second image)". Similar to the first image near the opening, the second image near the opening is an image captured of the exterior of the cargo compartment 2 within a predetermined distance of the opening. It should be noted that the second image near the opening may also include an image of the interior of the cargo compartment 2 (in which case, it can be said that the second image near the opening is both an example of the first image and an example of the second image).

[0034] The surveillance camera 6 transmits the images captured near the second opening to the cargo rate estimation device 100 (see reference). Figure 2 )send.

[0035] It should be noted that if the surveillance camera 6 has wireless communication capabilities, it can also directly transmit the image near the second opening wirelessly to the load factor estimation device 100. If the surveillance camera 6 does not have wireless communication capabilities, it can also transmit the image near the second opening via a wired connection to a wireless communication device (not shown) located at berth B, and the wireless communication device will then transmit the image near the second opening wirelessly to the load factor estimation device 100.

[0036] The above provides an explanation of vehicle V.

[0037] Next, use Figure 2 The cargo load factor estimation device 100 of this embodiment will be described. Figure 2 This is a block diagram illustrating an example configuration of the load factor estimation device 100.

[0038] As mentioned above, Figure 2 The load factor estimation device 100 shown is mounted on Figure 1 The vehicle shown is V.

[0039] Although not illustrated, the load factor estimation device 100 includes, as hardware, components such as a CPU (Central Processing Unit), a ROM (Read Only Memory) storing a computer program, and RAM (Random Access Memory). The functions of the devices described below are implemented by the CPU executing the computer program read from the ROM in the RAM. For example, the load factor estimation device 100 may also be implemented by an ECU (Electronic Control Unit).

[0040] like Figure 2 As shown, the load factor estimation device 100 has an identification unit 110 and an estimation unit 120.

[0041] The identification unit 110 acquires images of the interior of the cargo compartment from the in-cargo camera 4, images of the vicinity of the first opening from the rear camera 5, and images of the vicinity of the second opening from the surveillance camera 6. The images of the interior of the cargo compartment, the images of the vicinity of the first opening, and the images of the vicinity of the second opening (hereinafter collectively referred to as "each image") are, for example, real-time captured dynamic images.

[0042] Furthermore, the identification unit 110 identifies people and objects other than cargo in each image and tracks them to infer whether there are people and objects other than cargo inside the cargo compartment 2.

[0043] The "persons" mentioned here include, for example, the workers who perform loading or unloading operations in cargo compartment 2. Additionally, the "objects" mentioned here, other than cargo (hereinafter also simply referred to as "objects"), are the work tools (e.g., delivery trucks, double ladders, roller conveyors, etc.) used by the workers during the aforementioned operations.

[0044] The method for identifying and tracking people and objects in the identification unit 110 can employ known technologies. Examples of known technologies include image recognition technologies based on deep learning (e.g., see Japanese Patent Application Publication No. 2020-68008, Japanese Patent Application Publication No. 2020-204804, etc.) and image tracking technologies (e.g., see Japanese Patent Application Publication No. 2012-108798, Japanese Patent Application Publication No. 2020-91664, etc.), but are not limited to these.

[0045] When the identification unit 110 deduces that there are no people or objects other than cargo in the cargo compartment 2, the inference unit 120 infers the cargo load rate of the vehicle V based on the detection results of the depth sensor. The cargo load rate is the ratio of the volume of cargo configured in the cargo compartment 2 to the maximum cargo volume of the vehicle V.

[0046] The method for inferring the cargo load factor by the inference unit 120 can be based on known techniques. Examples of known techniques include, for instance, Japanese Patent Application Publication No. 2003-35527.<URL:https: / / creanovo.de / portfolio / wabco-cargocam / > ,<URL:https: / / www.ncos.co.jp / news / news_210113.html> The methods disclosed, but not limited to these.

[0047] It should be noted that the inference unit 120 may also output (send) information representing the inferred load factor to a specified device (not shown). Examples of specified devices include: a notification device (e.g., a display, a speaker, etc.) installed in the cab 1, and a computer (e.g., a server device on a network, etc.) installed outside the vehicle V.

[0048] The structure of the load factor estimation device 100 has been described above.

[0049] Next, use Figure 3 The operation of the load factor estimation device 100 will be explained. Figure 3 This is a flowchart illustrating an example of the operation of the load factor estimation device 100. Figure 3 The process shown can begin, for example, when vehicle V stops, or when vehicle V has stopped and door 3 is in the open state.

[0050] First, the recognition unit 110 acquires an image of the interior of the cargo compartment, an image near the first opening, and an image near the second opening (step S1).

[0051] Next, the identification unit 110 begins to identify and track people and objects other than cargo in each image (step S2). From this, the identification unit 110 infers whether there are people or objects other than cargo inside the cargo compartment 2 (step S3).

[0052] If there are people and objects other than cargo in the cargo compartment 2 (step S3: "Yes"), the above identification, tracking and inference are repeated.

[0053] If the identification unit 110 infers that there are no people or objects other than cargo in the cargo compartment 2 (step S3: "No"), the inference unit 120 performs an inference of the cargo load rate (step S4).

[0054] The operation of the load factor estimation device 100 has been explained above.

[0055] As described in detail above, the cargo load factor estimation device 100 of this embodiment is characterized in that it estimates the cargo load factor of vehicle V based on images of the cargo compartment, images near the first opening, and images near the second opening, inferring that there are no people or objects other than cargo in the cargo compartment 2.

[0056] Therefore, the load factor estimation device 100 of this embodiment can improve the accuracy of load factor estimation.

[0057] It should be noted that the present invention is not limited to the embodiments described above, and various modifications can be made without departing from its spirit. Hereinafter, examples of modifications will be described.

[0058] [Variation Example 1]

[0059] In this embodiment, the case in which the door 3 of the cargo compartment 2 is located at the rear (or "back") of the cargo compartment 2 is described as an example, but it is not limited to this.

[0060] For example, the opening of the cargo compartment 2 and the door 3 can also be located on the side of the cargo compartment 2 (specifically, at least one of the left and right sides). In this case, for example, a camera that captures images of the exterior and side of the vehicle V (specifically, near the outside of the opening) can be used instead of the rear camera 5.

[0061] [Variation Example 2]

[0062] In this implementation, the example is given where the vehicle V has a cargo box 2 that serves as a cargo platform, but the implementation is not limited to this.

[0063] Vehicle V can also be a vehicle that replaces the cargo box 2 with a flatbed platform (an example of a platform). In this case, for example, a camera installed in the cab 1 (e.g., outside and behind the cab 1) that captures images of the entire flatbed platform can be used instead of the camera 4 inside the cargo box. Alternatively, in this case, a camera installed on the flatbed platform that captures images of the area outside the flatbed platform at a predetermined distance from the entrance / exit of the flatbed platform can be used instead of the rear camera 5.

[0064] [Variation Example 3]

[0065] In this embodiment, the example is described using three types of images (e.g., an image of the interior of the cargo compartment, an image near the first opening, and an image near the second opening), but it is not limited to this and it is also possible to use only at least one of the three types of images.

[0066] First, let's explain the case where only images of the cargo compartment are used.

[0067] In this case, the inference unit 120 performs the inference of the cargo load rate at the time point when no person or object is identified in the image inside the cargo compartment.

[0068] Next, we will explain the case where only the image near the first opening is used.

[0069] Here, we will use the case where a worker is initially identified near the outside of the opening in the image near the first opening as an example. It should be noted that, as mentioned above, the image near the first opening does not include the image inside the cargo compartment 2.

[0070] If the worker identified by the identification unit 110 in the image near the first opening moves toward a predetermined direction of the opening (a direction known to the identification unit 110 on the image, hereinafter referred to as the "opening direction") and disappears from the image near the first opening, the worker is identified as being inside the cargo compartment 2. In this case, the load factor calculation performed by the inference unit 120 is not performed.

[0071] Subsequently, if the identification unit 110 detects the identified worker in the image near the first opening from the direction of the opening, it determines that the identified worker is not inside the cargo compartment 2. At this point, the inference unit 120 performs an inference of the cargo load rate.

[0072] In this way, even if only the image near the first opening, which does not include the image inside compartment 2, is used, it is possible to identify whether a person is inside compartment 2. It should be noted that the above description uses the case of one worker moving into compartment 2 as an example, but in the case of multiple workers moving into compartment 2, and in the case of workers moving into compartment 2 with work tools, the presence or absence of workers and work tools inside compartment 2 can be identified in the same way as described above.

[0073] Next, we will explain the case where only the image near the second opening is used.

[0074] When the image near the second opening includes an image of the interior of the cargo compartment 2, the identification unit 110 identifies and tracks people and objects in the image near the second opening, and the inference unit 120 infers the cargo load rate at the time point when it is assumed that these people are no longer in the cargo compartment 2. On the other hand, when the image near the second opening does not include an image of the interior of the cargo compartment 2, the operation of the identification unit 110 and the inference unit 120 is the same as in the case where only the image near the first opening is used.

[0075] It should be noted that the above description uses one of the following images as an example: an image of the interior of the cargo compartment, an image of the area near the first opening, and an image of the area near the second opening. However, it is also possible to use two of the following images: an image of the interior of the cargo compartment, an image of the area near the first opening, and an image of the area near the second opening.

[0076] As explained above, even when using only one camera (one captured image), the accuracy of cargo load estimation can be improved. However, using multiple cameras (multiple captured images), as in the embodiment, further improves the accuracy of recognition and tracking. Therefore, in the embodiment, the accuracy of cargo load estimation is higher than in this variation.

[0077] Furthermore, when using at least one of the images near the first opening and the second opening, the rear camera 5 and the surveillance camera 6, which are existing devices, can be utilized. In this case, since it is not necessary to install an in-cargo camera 4 inside the cargo compartment 2 (since only a depth sensor is required inside the cargo compartment 2), costs can be suppressed. That is, the cargo load rate can be accurately inferred using a low-cost and simple structure.

[0078] [Variation Example 4]

[0079] In this embodiment, the example described is of the load factor estimation device 100 being mounted on a vehicle V, but it is not limited thereto. For example, the load factor estimation device 100 may also be implemented by a computer (e.g., a server) located outside the vehicle V. In this case, for example, the images (e.g., images of the cargo compartment, images near the first opening, and images near the second opening) may be sent to the computer by a communication device (not shown) mounted on the vehicle V.

[0080] The above descriptions of the variations have been provided. It should be noted that the above variations can also be appropriately combined.

[0081] This application is based on Japanese Patent Application No. 2021-049749, filed on March 24, 2021, the contents of which are incorporated herein by reference.

[0082] Industrial applicability

[0083] The load factor estimation device of the present invention is useful for estimating the load factor of vehicles equipped with cargo boxes or cargo platforms.

[0084] Explanation of reference numerals in the attached figures

[0085] 1. Driver's cab

[0086] 2 cargo compartments

[0087] 3 doors

[0088] 4. Cameras inside the cargo compartment

[0089] 5. Rear camera

[0090] 6. Surveillance cameras

[0091] 100 Load Factor Determination Device

[0092] 110 Identification Department

[0093] 120 Inference Department

[0094] Berth B

[0095] V vehicle

Claims

1. A load factor estimation device, comprising: The identification unit acquires at least one of a second image and a first image captured inside the loading dock, and identifies and tracks people and objects other than cargo in the acquired images, thereby inferring whether the people and the objects other than cargo exist on the loading dock. The second image is an image captured of the exterior of the loading dock within a predetermined distance from the loading dock's entrance / exit. The inference unit infers the load factor of a vehicle equipped with the loading platform when there are no people or objects other than cargo on the loading platform.

2. The load factor estimation device as described in claim 1, wherein, The loading platform is a cargo box. The first image is an image captured by a camera installed inside the cargo compartment.

3. The load factor estimation device as described in claim 1, wherein, The second image is at least one of an image captured by an onboard camera that takes pictures of the exterior of the vehicle, and an image captured by a surveillance camera located at the location where the work is carried out on the cargo platform.

4. The load factor estimation device as described in claim 1, wherein, The person mentioned is the operator performing the work on the loading platform. The object other than the cargo is the work tool used in the operation.

Images