Mobile cargo compartment condition measuring device and cargo compartment condition measuring system

Abstract

To provide a mobile cargo compartment condition measurement device and cargo compartment condition measurement system that reliably monitors the loading status of trucks while keeping costs down. [Solution] The mobile cargo compartment condition measuring device 1 comprises a mobile terminal 10 having an imaging element 2 that acquires image information, a sensor element 3 that emits electromagnetic waves and acquires depth information from electromagnetic waves reflected from an object, and a calculation element connected to the imaging element and the sensor element. The imaging element 2 acquires image information of the truck's cargo compartment captured from one side in the vehicle width direction, the sensor element acquires depth information from one side of the cargo compartment toward the other side, and the calculation element, based on the image information and depth information, acquires the positions of the four sides consisting of the top edge, bottom edge, front edge and rear edge on the one side of the cargo compartment on the image information, determines that the area enclosed by the four sides is inside the cargo compartment, and determines the loading status inside the cargo compartment on the one side.

Description

Technical Field

[0001] The present invention relates to a technique for measuring the loading status of luggage in the cargo compartment of a truck.

Background Art

[0002] Trucks for collecting and delivering various pieces of luggage are provided with a cargo compartment for loading the luggage.

[0003] When collecting or delivering luggage, if an appropriate amount of luggage is not loaded into the cargo compartment of the truck and the empty space in the cargo compartment is larger than necessary, the number of truck trips required for collecting or delivering the luggage will be larger than the actual number of trips required.

[0004] In this case, the number of required trucks and the fuel cost of the trucks increase, and the number of required drivers also increases. As a result, not only problems occur in terms of cost and personnel procurement, but also problems occur in that the CO2 emissions increase.

[0005] Conventionally, in order to solve the above problems, for each individual truck loaded with luggage, the empty space in the cargo compartment is calculated and the result is fed back, so as to manage the route, trips, and load quantity per trip for collecting and delivering the luggage.

[0006] As a method for calculating the empty space in the cargo compartment of a truck, generally, a method is adopted in which an operator visually checks the amount of luggage, that is, the load quantity, in the cargo compartment of the truck, and calculates the empty space in the cargo compartment based on the load quantity visually checked.

[0007] However, when an operator visually checks the load quantity, due to variations in the proficiency of each operator, a large variation among operators occurs between the load quantity confirmed by the operator and the actual load quantity. Therefore, conventionally, it has been difficult to sufficiently increase the loading rate of trucks and efficiently carry out the transportation of luggage.

[0008] To efficiently transport goods by maximizing truck loading capacity, it is necessary to reduce the difference between the amount of cargo confirmed by the worker and the actual amount of cargo. As mentioned above, the difference between the amount of cargo confirmed by the worker and the actual amount of cargo is largely due to variations in the skill level of each worker. Therefore, reducing the involvement of workers in the process of assessing the loading status inside the cargo compartment of a truck may reduce the above-mentioned difference.

[0009] One method to reduce the involvement of workers in the task of understanding the loading status inside the cargo compartment of a truck is to determine the amount of cargo and available space in the cargo compartment based on images of the cargo compartment (see, for example, Patent Document 1). Patent Document 1 discloses a technique for calculating the available space in a truck's cargo compartment by imaging the cargo compartment with multiple CCD cameras installed in the cargo compartment and analyzing the obtained images with an in-vehicle information processing device including a microcomputer. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Publication No. 2001-334864 [Overview of the Initiative] [Problems that the invention aims to solve]

[0011] As described above, when the cargo compartment of a truck is imaged and the cargo volume and available space are calculated based on the obtained image information, the involvement of workers in the task of understanding the loading situation inside the truck's cargo compartment can be reduced. However, existing technologies, such as those introduced in Patent Document 1, require a large number of cameras for the cargo compartment of a single truck, which is not ideal from a cost perspective. Reducing the number of cameras makes it difficult to image the entire cargo compartment, leading to problems in reliably understanding the cargo volume and available space inside the cargo compartment, or in other words, problems in reliably understanding the loading situation inside the truck's cargo compartment.

[0012] This invention has been made in view of the above circumstances, and aims to solve the problem of providing a technology that can reliably grasp the loading status inside the cargo compartment of a truck while suppressing the rise in costs. [Means for solving the problem]

[0013] The mobile cargo compartment condition measuring device of the present invention, which solves the above problems, A mobile terminal comprising an imaging element for acquiring image information, a sensor element for emitting electromagnetic waves and acquiring depth information from the electromagnetic waves reflected by the object, and a calculation element connected to the imaging element and the sensor element, The aforementioned imaging element acquires image information of the truck's cargo area from one side in the vehicle width direction. The sensor element acquires the depth information of the cargo compartment from one side to the other side. The aforementioned calculation element is Based on the aforementioned image information and depth information, a 3D mesh including positional information of the object is created on the image information. In the information of the aforementioned 3D mesh, The positions of the four sides of the cargo compartment, consisting of the top edge, bottom edge, front edge, and rear edge, on one side of the cargo compartment are obtained. The area enclosed by the four sides is determined to be inside the cargo compartment. Based on the above determination, the target area within the cargo compartment to be used for calculating the amount of cargo and / or empty space is identified, A surface located within the aforementioned target area and facing the imaging element is identified as the cargo, This is a mobile cargo compartment condition measuring device that determines the loading status of the cargo inside the cargo compartment on one side.

[0014] The cargo compartment condition measurement system of the present invention, which solves the above problems, A cargo compartment condition measurement system for measuring the loading conditions inside the cargo compartment of a truck, using a terminal having an imaging element that acquires image information, a sensor element that emits electromagnetic waves and acquires depth information from electromagnetic waves reflected from an object, and a calculation element connected to the imaging element and the sensor element, The imaging element acquires the image information obtained by imaging the cargo area from one side in the vehicle width direction. The sensor element acquires the depth information of the cargo compartment from one side to the other. The aforementioned calculation element, Based on the aforementioned image information and depth information, a 3D mesh including positional information of the object is created on the image information. In the information of the aforementioned 3D mesh, The positions of the four sides of the cargo compartment, consisting of the top edge, bottom edge, front edge, and rear edge on one side, are obtained, and the area enclosed by these four sides is determined to be inside the cargo compartment. Based on the above determination, the target area within the cargo compartment to be used for calculating the amount of cargo and / or empty space is identified, A surface located within the aforementioned target area and facing the imaging element is identified as the cargo, This is a cargo compartment condition measurement system that determines the loading status of the cargo inside the cargo compartment on one side. [Effects of the Invention]

[0015] According to the mobile cargo compartment status measurement device and the cargo compartment status measurement system of the present invention, it is possible to reliably grasp the loading status in the cargo compartment of a truck while suppressing the soaring of costs.

Brief Description of the Drawings

[0016] [Figure 1] It is an explanatory diagram schematically showing a state in which the loading status in the cargo compartment of a truck is determined by the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 1. [Figure 2] It is an explanatory diagram schematically explaining the mobile terminal used in the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 1. [Figure 3] It is an explanatory diagram schematically explaining the mobile terminal used in the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 1. [Figure 4] It is an explanatory diagram schematically showing a state in which the loading status in the cargo compartment of a truck is determined by the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 1. [Figure 5] It is an explanatory diagram schematically showing a state in which the loading status in the cargo compartment of a truck is determined by the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 1. [Figure 6] It is a flowchart schematically explaining the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 1. [Figure 7] It is an explanatory diagram schematically explaining a truck at a loading and unloading base and its cargo compartment. [Figure 8] It is an explanatory diagram schematically showing the image information obtained by imaging a truck at a loading and unloading base and its cargo compartment with an imaging element. [Figure 9] It is an explanatory diagram schematically showing a state in which the loading status in the cargo compartment of a truck is determined by the mobile cargo compartment status measurement device and the cargo compartment status measurement system of Example 2. [Figure 10]This is a schematic diagram illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck. [Figure 11] This is a schematic diagram illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck. [Figure 12] This is a schematic diagram illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck. [Figure 13] This is a schematic diagram illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck. [Figure 14] This is a schematic diagram illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck. [Figure 15] This is a schematic diagram illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck. [Modes for carrying out the invention]

[0017] The following describes the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention with specific examples.

[0018] In this specification, "truck" refers to a cargo vehicle having a cargo compartment capable of carrying goods, and is a concept that includes wagons and vans with cargo compartments.

[0019] Furthermore, the term "loading status in the cargo compartment of a truck" as used herein refers to the state of the cargo compartment, specifically the extent to which goods are loaded in the cargo compartment at that time, or the extent to which more goods can be loaded in the cargo compartment. The indicators used are not particularly limited. For example, one or more indicators selected from the following may be used to describe the loading status of a truck: the volume of space occupied by goods loaded in the cargo compartment of the truck, the sum of the volumes of goods loaded in the cargo compartment of the truck, the number of goods loaded in the cargo compartment of the truck, the volume of remaining empty space in the cargo compartment of the truck, the sum of the volumes of goods that can be loaded in the empty space, and the number of goods that can be loaded in the empty space.

[0020] Unless otherwise specified, the numerical range "x~y" described herein includes a lower limit x and an upper limit y. Furthermore, a numerical range can be constructed by arbitrarily combining these upper and lower limits, as well as the numerical values ​​listed in the embodiments. Additionally, any numerical values ​​selected from within the numerical range can be used as the upper and lower limits.

[0021] As previously mentioned, existing technologies, such as those described in Patent Document 1, make it difficult to reliably and cost-effectively ascertain the loading conditions inside a truck's cargo compartment. This is due to the following reasons.

[0022] First, as mentioned earlier, when numerous cameras are installed in the cargo compartment of a single truck, the entire cargo compartment can be imaged, and the amount of cargo and available space in the cargo compartment can be reliably determined based on the obtained images. However, this method involves very high costs for installing and maintaining the cameras, so it cannot be said that it is a low-cost way to understand the loading situation inside the cargo compartment of a truck.

[0023] Reducing the number of cameras makes it difficult to capture images of the entire cargo compartment, leading to an increased discrepancy between the calculated cargo volume and empty space based on the images obtained and the actual cargo volume and empty space. Consequently, it becomes difficult to reliably determine the cargo volume and empty space in the cargo compartment, and consequently, it becomes difficult to reliably determine the loading conditions inside the truck's cargo compartment.

[0024] By installing the camera outside the cargo area and increasing the distance between the camera and the cargo area, it becomes possible to image a relatively large area of ​​the cargo area with a single camera. However, when imaging the cargo area with a camera installed outside the cargo area, there is a high possibility that objects other than the cargo area will be captured in the resulting image.

[0025] The process of imaging the cargo compartment to determine the amount of cargo and available space is preferably performed at the time of loading or unloading cargo. However, loading and unloading points, specifically production sites, logistics centers, and customer warehouses, see a large volume of trucks coming and going, and a large volume of goods moving back and forth.

[0026] At such loading and unloading points, as shown in Figure 7, for example, another truck 91 may be parked next to the truck 90 that is to be imaged. In this case, if the cargo compartment 95 is imaged with a camera (not shown) installed on the outside of one side of the cargo compartment 95, the resulting image will include not only the cargo compartment 95 of the truck 90 being imaged and the cargo 97 inside the cargo compartment 95, as well as the cargo compartment 96 of the other truck 91 parked next to the truck 90, and the cargo 98 inside the cargo compartment 96, as shown in Figure 8.

[0027] In such images, it is extremely difficult to distinguish between the target area 95I of the cargo compartment 95 of the truck 90 being imaged, the area 95E of the cargo compartment 95 other than the target area 95I, and the cargo compartment 96 of the truck 91 that is not being imaged. Furthermore, in this case, it is also extremely difficult to determine whether the cargo captured in the image is cargo 97I located in the target area 95I of the cargo compartment 95, cargo 97E located in the area 95E other than the target area 95I of the cargo compartment 95, or cargo 98 located in the cargo compartment 96. Therefore, in such cases, it becomes difficult to reliably determine the amount of cargo and available space in the cargo area based on images, resulting in a problem where it is difficult to reliably understand the loading situation inside the cargo area.

[0028] As a result of diligent research, the inventors of the present invention have found that by using a sensor element that acquires depth information in addition to an imaging element such as a camera that acquires image information, it is possible to reliably distinguish between cargo in the target area and other areas, specifically, areas of the cargo compartment other than the target area as described above, and cargo in the cargo compartment of a truck that is not the target of imaging.

[0029] The sensor element emits electromagnetic waves and acquires depth information from the electromagnetic waves reflected by the target object. As depth information, such a sensor element can detect the distance between the cargo (the object from which the electromagnetic waves are reflected) and the sensor element. Therefore, by using this depth information in addition to image information, it becomes possible to reliably distinguish between cargo in the target area 95I and cargo in other areas, and to reliably understand the loading status of the truck.

[0030] Furthermore, in the mobile cargo compartment condition measuring device of the present invention, the imaging element that acquires image information and the sensor element that acquires depth information are mounted on a mobile terminal.

[0031] In recent years, it has become common practice to provide truck drivers with mobile devices such as smartphones. If a commercially available, general-purpose device equipped with imaging and sensor elements is selected as the mobile device, the only additional costs required for the mobile cargo compartment condition measuring device of the present invention will be the costs of developing and distributing the application corresponding to the calculation element. Furthermore, even when a mobile terminal is a proprietary product, general-purpose components can be used as at least some of the parts in that mobile terminal, thus making it possible to suppress cost increases in this case as well. Therefore, the mobile cargo compartment condition measuring device of the present invention makes it possible to reduce the costs required for the introduction and maintenance of the mobile cargo compartment condition measuring device.

[0032] Furthermore, in the cargo compartment condition measurement system of the present invention, the terminal having the imaging element, sensor element, and calculation element is not specified as a mobile terminal; for example, the terminal may be a fixed terminal. Even in such a cargo compartment condition measurement system of the present invention, the effect of the present invention remains the same: it is possible to reliably distinguish between cargo in the target area 95I and cargo in other areas based on depth information in addition to image information, thereby enabling a reliable understanding of the loading status of the truck.

[0033] Furthermore, in the cargo compartment condition measurement system of the present invention, a mobile terminal can be used as the terminal having an imaging element, a sensor element, and a calculation element. In this case, similar to the mobile cargo compartment condition measurement device of the present invention described above, the costs required for the introduction and maintenance of the cargo compartment condition measurement system of the present invention can be reduced.

[0034] Based on the above, it can be said that the mobile cargo compartment condition measuring device of the present invention makes it possible to reliably grasp the loading conditions inside the cargo compartment of a truck while suppressing cost increases. Furthermore, it can be said that the cargo compartment condition measuring system of the present invention also makes it possible to reliably grasp the loading conditions inside the cargo compartment of a truck while suppressing cost increases by using a mobile terminal as the terminal.

[0035] The mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention will be described below for each component.

[0036] Furthermore, when a mobile terminal is used as the terminal for measuring the cargo compartment conditions of the present invention, it can be said that the mobile cargo compartment condition measuring device of the present invention is used to grasp the loading conditions inside the cargo compartment of a truck. For this reason, components with the same name are common to both the cargo compartment condition measuring system and the mobile cargo compartment condition measuring device of the present invention. In this specification, the descriptions of common components will serve the same purpose as the descriptions of each component in the mobile cargo compartment condition measuring device and each component in the cargo compartment condition measuring system of the present invention.

[0037] The cargo compartment condition measurement system of the present invention is a cargo compartment condition measurement system for measuring the loading conditions inside the cargo compartment of a truck using a terminal having an imaging element, a sensor element, and a calculation element. As previously mentioned, the device in question may be a mobile device or a fixed-line device. The mobile cargo compartment condition measuring device of the present invention comprises a mobile terminal having an imaging element, a sensor element, and a calculation element.

[0038] The terminal and mobile terminal have an imaging element, a sensor element, and a computing element.

[0039] The imaging element only needs to acquire image information, and this image information may be video or still images. The imaging element may be capable of capturing only still images, only video, or both video and still images.

[0040] For the imaging element, it is sufficient to acquire at least one image of the truck's cargo compartment, which is the object whose loading status should be monitored, at the same loading / unloading location. However, it is more preferable to acquire multiple images over time. This is because if the worker acquiring the image information is different from the worker loading or unloading the cargo onto the truck, it is possible that the loading and unloading of cargo may not be completed at the time the image information is acquired.

[0041] If the loading and unloading of cargo is not yet complete when the image information is acquired, there is a risk of discrepancies between the cargo volume shown in the image information and the actual cargo volume loaded in the cargo compartment. However, by acquiring multiple images over time, the probability of acquiring images after the loading and unloading of cargo has been completed increases, making it possible to reduce or eliminate such discrepancies.

[0042] The amount of cargo in the cargo compartment is highest for collection services immediately after unloading begins, specifically immediately after the truck's wings open, and highest for delivery services immediately after loading begins, specifically just before the truck's wings close.

[0043] Therefore, in the case of collection services, it is preferable that the above-mentioned multiple image information includes image information taken immediately after the opening of the truck's wings. Specifically, it is particularly preferable that the image information includes image information taken within 180 seconds, 120 seconds, 60 seconds, or 30 seconds after the opening of the truck's wings. Furthermore, in the case of delivery shipments, the image information preferably includes images taken immediately before the truck's wings begin to close. Specifically, it is particularly preferable to include images taken within 180 seconds, 120 seconds, 60 seconds, or 30 seconds before the truck's wings begin to close.

[0044] Furthermore, if the worker acquiring the image information is the same person who loads or unloads the cargo onto the truck, the same worker only needs to acquire the image information after the loading and unloading is complete. In this case, only one image is needed, and the timing of acquiring the image information does not need to differ significantly from the completion of the truck's wing opening or closing operations.

[0045] The sensor element only needs to emit electromagnetic waves and acquire depth information from the electromagnetic waves reflected by the object. A known structure using infrared, laser waves, millimeter waves, etc. as electromagnetic waves can be appropriately selected as needed.

[0046] For example, it is preferable to use a sensor called LiDAR (Light Detection And Ranging) as the sensor element. LiDAR is a device that emits electromagnetic waves such as infrared rays or laser waves and measures the time it takes for the emitted electromagnetic waves to reflect off the target object and return, thereby acquiring information such as the distance, direction, and shape of the object from the sensor. Depending on the type of electromagnetic wave used, it is also called infrared LiDAR, laser LiDAR, etc.

[0047] In this specification, the information such as the distance, direction, and shape of the object measured by the sensor element as described above will be referred to as depth information. While two-dimensional positional information can be obtained from image information acquired by imaging elements, it is possible to obtain three-dimensional positional information by adding depth information acquired by sensor elements to that image information.

[0048] The timing at which depth information is acquired by the sensor element may be different from or the same as the timing at which image information is acquired by the imaging element. However, in order to reliably understand the loading situation inside the cargo compartment by comparing the two-dimensional information acquired by the imaging element with the depth information acquired by the sensor element, it is preferable that the timing at which depth information is acquired by the sensor element is the same as the timing at which image information is acquired by the imaging element. Here, "same" means that the time difference between the two is within 1 / 10 of a second. Preferably, the time difference between the two is within 1 / 20 of a second, 1 / 30 of a second, or 1 / 60 of a second.

[0049] The computing element is connected to the imaging element and the sensor element, and can perform processing to determine the loading status inside the cargo compartment based on the image information obtained by the imaging element and the depth information obtained by the imaging element. Its structure and configuration are not particularly limited. Examples of such computing elements include smartphones, tablets, and computers equipped with a computing control device such as a CPU or GPU, memory such as RAM, and various storage devices such as internal storage or SD cards. The mobile terminal in this invention may also be equipped with one or more other components selected from various other components that a mobile terminal may have, such as a touch panel, liquid crystal module, vibration generation module, voice module, battery, antenna, wireless LAN module, GPS module, gyro sensor, etc. A fixed terminal may also be equipped with similar components.

[0050] The computational elements may be connected to the imaging elements and sensor elements by wires or by wireless connections.

[0051] In the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention, the imaging element acquires image information of the truck's cargo compartment captured from one side in the vehicle width direction. In other words, the imaging element acquires image information of the truck's cargo compartment captured from the driver's side or the passenger's side.

[0052] Furthermore, in the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention, the sensor element acquires depth information of the cargo compartment from one side toward the other side. The image information and depth information can be said to be information of the same cargo compartment and information directed from the same side toward the same direction of the cargo compartment.

[0053] The calculation element performs a process to determine the loading status inside the cargo compartment based on the above image information and depth information. This process may be image processing. Specifically, the calculation element obtains the positions of the four sides consisting of the top edge, bottom edge, front edge, and rear edge on one side of the cargo compartment in the image information. Then, it determines that the area enclosed by these four sides is inside the cargo compartment in the image information. This area enclosed by these four sides can also be called the target area 95I within the cargo compartment for which we want to calculate the amount of cargo and empty space, as described above. By doing this, the cargo area of ​​the truck, which is the target of the load situation assessment, can be identified, and cargo located outside the area enclosed by the four sides mentioned above—in other words, cargo that should not be counted as cargo—can be reliably excluded. Details of the four sides will be explained in detail in the section on examples below.

[0054] In the following, trucks whose loading status should be monitored may be referred to as "target trucks" as needed. Trucks that are not target trucks may be referred to as "non-target trucks."

[0055] Furthermore, the calculation element determines the loading status within the area determined to be inside the cargo compartment, as described above, and on one side as described above. Since the calculation element performs image processing based on image information and depth information, the determination of the loading status is performed by using both the two-dimensional image information acquired by the imaging element and the three-dimensional depth information acquired by the sensor element, as described above.

[0056] By using information about the rear of the cargo area in addition to image information as the basis for image processing, it is possible to determine whether cargo that can be identified as being inside the cargo area in the image information is located within a predetermined range from one end of the cargo area.

[0057] For example, by determining the loading status within the predetermined range of the total length in the vehicle width direction of the cargo compartment, it is possible to reliably exclude cargo in the cargo compartments of non-target trucks—in other words, cargo in areas that should not be counted as cargo—from the items whose loading status should be assessed. This allows for the reliable detection of only the cargo in the target truck's cargo compartment, making it possible to reliably understand the loading status within the cargo compartment of the target truck.

[0058] Furthermore, if, for example, the driver's side of the truck is one of the aforementioned sides, the loading status may be determined by defining the half-area of ​​the cargo compartment located on the driver's side as the predetermined range. In this case, cargo located on the passenger side of the cargo compartment of the target truck, and cargo in the cargo compartment of other trucks, can be reliably excluded. In this case, in addition to determining the loading status in the cargo compartment on one side as described above, determining the loading status in the cargo compartment on the other side allows for reliable detection of cargo in the cargo compartment of the target truck across the entire width of the vehicle, making it possible to understand the loading status in the cargo compartment of the target truck with greater reliability.

[0059] When determining the loading status by defining the area of ​​the cargo compartment as either half of the area located on one side or the other half of the area located on the other side in the vehicle width direction, it is preferable to use a part of the truck, including the cargo compartment, as a criterion for distinguishing between one side and the other side in the vehicle width direction. This is because the dimensions of the cargo compartment and its length in the vehicle width direction may vary depending on the truck model and its year of manufacture. Hereafter, as needed, the area located on one side of the cargo compartment in the vehicle width direction will be referred to as the "one-side half area," and the area located on the other side of the cargo compartment in the vehicle width direction will be referred to as the "other-side half area."

[0060] For example, if a truck is a wing truck having a first wing that can be opened and closed to cover one half of the area on one side, and a second wing that can be opened and closed to cover the other half of the area on the other side, then the first wing or the second wing can be used as a criterion to distinguish between one side and the other side in the vehicle width direction within the cargo area.

[0061] In a typical wing truck, the first and second wings rotate around the center of the truck in the width direction, and can open and close to cover one half of the area or the other half of the area. Therefore, by using the root portion of the first wing as a boundary and identifying the area on one side of the root portion as the one-side half region, and by using the root portion of the second wing as a boundary and identifying the area on the other side of the root portion as the one-side half region, it is possible to distinguish between the one-side half region and the other-side half region with high accuracy and ease.

[0062] The base portions of the first and second wings can be more easily identified when the cargo compartments of the first and second wings are open. Therefore, it is particularly preferable to define the area on one side of the base portion of the first wing as the one-side half region when the cargo compartment is open, and to define the area on one side of the base portion of the second wing as the one-side half region when the cargo compartment is open.

[0063] In a typical wing van, the base of the first wing and the base of the second wing are adjacent in the vehicle width direction. Therefore, it is possible to recognize the base of the first track and the base of the second track as a single base, and to identify the area on one side of this single base as the one-side half area, and the area on the other side as the other-side half area.

[0064] As previously described, the imaging element captures images of the truck's cargo compartment from one side and acquires image information. Therefore, it is preferable for the calculation element to determine the loading status of the cargo compartment as the loading status of the cargo compartment, i.e., the one-sided 1 / 2 region that is covered by the first wing and can be opened and closed. In addition, it is also preferable for the imaging element to capture images of the truck's cargo compartment from the other side and acquire image information, and for the calculation element to determine the loading status of the other-sided 1 / 2 region, which is the other-sided region of the cargo compartment, as the second loading status of the cargo compartment. As previously described, this makes it possible to reliably detect the cargo in the target truck's cargo compartment across the entire width of the vehicle, and to understand the target truck's loading status with greater reliability.

[0065] Incidentally, the mobile cargo compartment condition measuring device of the present invention uses the imaging element, sensor element, and calculation element mounted on a mobile terminal as described above. As the mobile terminal, a commercially available general-purpose product equipped with the imaging element and sensor element may be selected, or a dedicated product for the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention may be used. In either case, by using a mobile terminal to grasp the loading conditions inside the cargo compartment of a truck, it is possible to reduce the cost required for the device for grasping the loading conditions inside the cargo compartment of a truck. The same applies to the terminal used in the cargo compartment condition measuring system.

[0066] In the mobile cargo compartment condition measuring device of the present invention, the worker who acquires image information and depth information using a mobile terminal may be a truck driver or a worker other than a driver working at the loading / unloading base. However, since the loading status inside the cargo compartment of each truck needs to be understood, and a very large number of trucks come and go at the loading / unloading base, it can be said that it is reasonable for the truck driver to acquire image information and depth information using a mobile terminal themselves. The same applies to the worker who acquires image information and depth information using a terminal in the cargo compartment condition measuring system of the present invention.

[0067] Because the mobile terminal equipped with imaging elements, sensor elements, and computing elements can change position freely, the positional relationship between the imaging and sensor elements mounted on the mobile terminal and the cargo compartment of the target truck will differ each time a measurement is taken.

[0068] For example, if the mobile terminal is tilted horizontally or vertically, the target area 95I in the cargo compartment from which the amount of cargo volume and empty space is to be calculated will be displayed tilted or distorted in the image information. If the tilt or distortion of such target area 95I is large, the reliability of the truck's loading status as determined by the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention may decrease.

[0069] To suppress or resolve the above-mentioned problems caused by the mobile terminal tilting horizontally and vertically, it is preferable to suppress the tilt of the mobile terminal itself when acquiring image information and depth information of the cargo area. Alternatively, it is also preferable to correct the image information and depth information of the cargo area according to the tilt of the mobile terminal itself.

[0070] One method to suppress the tilt of the mobile device itself when acquiring image information and depth information of the cargo area is to correct the tilt of the mobile device using external or internal elements.

[0071] One method for correcting a mobile device's tilt caused by external factors is to fix the mobile device in a predetermined position and facing a predetermined direction. Specifically, it is preferable to install fixing devices for securing mobile terminals in the loading and unloading area at the loading and unloading base, where trucks are parked and cargo is loaded and unloaded. When acquiring image and depth information of the cargo area, fixing a mobile terminal to this fixing device allows the mobile terminal to be corrected to the proper position and orientation, suppressing tilt of the mobile terminal in the horizontal and vertical directions, and ultimately enabling a more reliable understanding of the truck's loading status.

[0072] One method for correcting the tilt of a mobile device using internal elements is to utilize the angular velocity sensor (so-called gyroscope) and accelerometer installed in the mobile device.

[0073] These sensors can detect the tilt of a mobile device, and based on the detection results, a message can be displayed on the mobile device indicating that it is tilted, allowing the operator to correct the tilt of the mobile device based on the displayed message.

[0074] A mobile device may visually provide the above-mentioned warning using an LCD module or the like installed in the mobile device, tactilely provide the above-mentioned warning using a vibration generating module or the like installed in the mobile device, or audibly provide the above-mentioned warning using an audio module or the like installed in the mobile device. Furthermore, two or more of these may be used in combination.

[0075] The following describes the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of the present invention with specific examples.

[0076] (Example 1) The cargo compartment condition measurement system of Example 1 uses a mobile terminal as the terminal. Figures 1, 4, and 5 are schematic diagrams illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 1 determine the loading conditions inside the cargo compartment of a truck. Figures 2 and 3 are schematic diagrams illustrating the mobile terminal used in the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 1. Figure 6 is a schematic flowchart illustrating the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 1.

[0077] For more details, Figure 1 schematically illustrates how a driver, acting as the operator, captures images of the truck's cargo area and acquires image and depth information. Figure 2 schematically illustrates how a mobile terminal is tilted from one side to the other, and Figure 3 schematically illustrates how a mobile terminal is tilted in the direction of vehicle travel. Figure 4 schematically illustrates the target area and the area outside the target area in the cargo compartment of a truck, and Figure 5 illustrates the four sides of the truck's cargo compartment in image information.

[0078] The mobile cargo compartment condition measuring device 1 of Example 1 comprises a mobile terminal 10 having an imaging element 2, a sensor element 3, and a calculation element 4. The mobile terminal 10 is a smartphone.

[0079] The imaging element 2 is a camera mounted on a smartphone, i.e., a mobile device 10, and is capable of acquiring still images and videos as image information. Sensor element 3 is an infrared LiDAR mounted on the mobile terminal 10, which emits infrared radiation as electromagnetic waves and acquires depth information by measuring the time it takes for the emitted infrared radiation to reflect off an object and return.

[0080] The computing element 4 is comprised of a part of the mobile terminal 10, which includes an arithmetic control unit, memory, and internal storage.

[0081] Calculation element 4 includes an application for determining the loading status, as well as a communication application for communicating the loading status to a database for centrally managing the loading status of each track 9. The imaging element 2 and the sensor element 3 are connected to the calculation element 4 by wire and wirelessly to the database mentioned above.

[0082] The loading and unloading base 99B has numerous loading and unloading areas 99A. As shown in Figure 1, each loading and unloading area 99A is a section where one truck 9 can be parked, and is enclosed by lines drawn on the building floor. When truck 9 enters the premises, it is inside the loading and unloading area 99A, and when it leaves the premises, it is outside the loading and unloading area 99A.

[0083] As shown in Figure 1, the driver 99D uses the imaging element 2 and sensor element 3 mounted on the mobile terminal 10 to acquire image information of the cargo compartment 95 of the target truck 90, which is the object to be imaged, from one side in the vehicle width direction (right side, driver's side in Example 1), and depth information from that one side in the vehicle width direction to the other side (right side to left side, driver's side to passenger side in Example 1). In Example 1, the acquisition of image information and depth information is performed simultaneously, and furthermore, the image information and depth information are acquired multiple times at 60 frames per second (60 FPS).

[0084] The calculation element 4 is also connected to the angular velocity sensor (not shown), acceleration sensor (not shown), and liquid crystal module (not shown) of the mobile terminal 10. The angular velocity sensor and acceleration sensor detect the tilt of the mobile terminal 10, and if the mobile terminal 10 is tilted in one direction from one side to the other as shown in Figure 2, or tilted in the direction of vehicle travel as shown in Figure 3, a message to that effect is displayed on the liquid crystal screen 19.

[0085] Specifically, in the mobile cargo compartment condition measuring device 1 of Example 1, two lines are displayed on the LCD screen 19: a reference line (not shown) that serves as a reference for the posture of the mobile terminal 10, and a measured line (not shown) that represents the actual posture of the mobile terminal 10. The operator can adjust the posture of the mobile terminal 10 to an appropriate state by correcting the tilt of the mobile terminal 10 in the one-to-the-other direction and in the direction of vehicle travel so that these two lines overlap.

[0086] The details of the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1 will be described below, mainly based on the flowchart in Figure 6. In Example 1, the truck driver 99D operates the mobile cargo compartment condition measuring device 1 and the cargo compartment condition measuring system as an operator.

[0087] Driver 99D, who has parked truck 9 in the loading / unloading area 99A of loading / unloading base 99B, first operates the mobile terminal 10 to launch an application for determining the loading status inside the cargo compartment 95 of the truck 9, i.e., the target truck 90 (S1 in Figure 6).

[0088] When the above application is launched, the load measurement screen opens and load measurement begins (S2).

[0089] After confirming that the load measurement screen is open, driver 99D stands on one side of the cargo compartment 95 of the target truck 90 (right side in Example 1) and operates the mobile terminal 10 while facing the other side (left side in Example 1) (Figure 1). At this time, as described above, the tilt of the mobile terminal 10 is corrected to the appropriate position (Figures 2 and 3). Then the imaging element 2 and sensor element 3 operate, and image information and depth information are acquired (S3 in Figure 6). The image information acquired at this time is an image of the target truck 90, centered on the cargo compartment 95, taken from the right side, i.e., one side in the vehicle width direction. The rear view information acquired at this time is information directed from the right side, i.e., one side in the vehicle width direction, of the target truck 90, centered on the cargo compartment 95, towards the left side, i.e., the other side.

[0090] In the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, the right-hand side of the cargo compartment 95 of the target truck 90, which is one half region, is defined as the target region 95I, and the left-hand side of the cargo compartment 95, which is the other half region, is defined as the area outside the target region 95E. Naturally, areas outside the cargo compartment 95 of the target truck 90, such as the cargo compartment 96 of other trucks 91, are also outside the target region.

[0091] As shown in Figure 8 described above, the image information obtained in S3 includes not only the cargo 97I in one half of the region, i.e., the target region 95I, but also cargo 97E in the other half of the region, i.e., outside the target region 95E, and cargo 98 in the cargo compartment 96 of the non-target truck 91. Therefore, in the following steps, information on cargo 97E, 98, etc., located in regions other than the target region 95I will be removed from the image information.

[0092] First, the calculation element 4 creates a point cloud on the image information that indicates the position of the object reflected by the infrared light emitted by the sensor element 3, based on the depth information described above (S4). The point cloud created on the image information includes two-dimensional position information in the direction of vehicle movement and vertical direction for objects within the imaging range of the imaging element 2, as well as position information in the vehicle width direction for objects within the detection range of the sensor element 3.

[0093] The calculation element 4 then creates a three-dimensional mesh, or 3D mesh, on the image information based on the image information and depth information (S5). The 3D mesh also includes not only two-dimensional position information in the direction of vehicle movement and vertical direction for objects within the imaging range of the imaging element 2, but also position information in the vehicle width direction for objects within the detection range of the sensor element 3.

[0094] The 3D mesh shown above has the position information of the point cloud obtained in S4 overlaid on it. Since the point cloud indicates the position of objects that reflect the infrared light emitted by sensor element 3, the position information of the luggage 97I located in the target area 95I can be determined from the position information of the point cloud in that target area 95I.

[0095] In the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, the target truck 90 is a wing truck. As shown in Figure 4, the wing truck has a first wing FW that can be opened and closed to cover one half of the area (target area 95I), which is the target area 95I, and a second wing SW that can be opened and closed to cover the other half of the area (non-target area 95E).

[0096] The base portion of the first wing FW and the base portion of the second wing SW are each located in the center of the vehicle width direction of the target truck 90. ​​Therefore, in the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Embodiment 1, the area on one side of the base portion of the first wing FW is identified as the area including the one-side 1 / 2 area (target area 95I), and the other area is identified as the area not including the one-side 1 / 2 area (i.e., the other-side 1 / 2 area, non-target area 95E). Furthermore, the region identified at this time, located on one side of the base of the first wing FW, also includes the region above the cargo compartment 95, the region below the cargo compartment 95, the region ahead of the cargo compartment 95 in the direction of vehicle travel, and the region ahead of the cargo compartment 95 in the direction of vehicle travel.

[0097] In the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, the calculation element 4 acquires positional information of the root portion of the first wing FW from the 3D mesh obtained in S5, and based on this, identifies a region including one side 1 / 2 region (target region 95I). Then, from the positional information of the point cloud in that region, it identifies cargo that is inside the region and cargo that is outside the region, and excludes information of cargo that is outside the region.

[0098] The calculation element 4 then obtains the positions of the four edges, consisting of the top edge, bottom edge, front edge, and rear edge, on one side of the cargo compartment 95 based on the 3D mesh described above (S6-S8).

[0099] The calculation element 4 first performs a ceiling determination to obtain the position of the upper edge of the cargo compartment 95 (S6). As shown in Figure 5, the 3D mesh includes image information of the cargo compartment 95 as well as image information of the target truck 90 other than the cargo compartment 95 and its surroundings.

[0100] The ceiling determination is a process of obtaining the position of the upper edge US of the cargo compartment 95, in other words, the ceiling of the cargo compartment 95, in order to distinguish the image information of the cargo compartment 95 from the image information of other objects above the cargo compartment 95. Specifically, the calculation element 4 scans the 3D mesh from top to bottom and detects edges that extend horizontally on the 3D mesh. It then extracts the first edge detected that is considered to have an actual length of 6m or more, and obtains its positional information as the upper edge US of the cargo compartment 95 (in other words, the ceiling of the cargo compartment 95).

[0101] The calculation element 4 then performs a tailgate determination to obtain the position of the lower edge of the cargo compartment 95 (S7). The tailgate LW is a plate-shaped component that surrounds the cargo compartment 95 to prevent cargo from falling out, and the tailgate LW is opened downwards when loading and unloading cargo. In the 3D mesh shown in Figure 5, the tailgate LW is located below the cargo compartment 95.

[0102] The tailgate detection process involves obtaining the position of the lower edge LS of the cargo compartment 95 in order to distinguish between the image information of the cargo compartment 95 and the image information of other areas below the cargo compartment 95. Specifically, the calculation element 4 scans the 3D mesh from bottom to top and detects edges that extend horizontally on the 3D mesh. Then, it extracts the edge that is detected after the upper edge US mentioned above and is considered to have an actual length of 6m or more, and obtains its positional information as the lower edge LS of the cargo compartment 95 (in other words, the upper edge of the tailgate LW).

[0103] Furthermore, the leading edge of the first wing FW and the leading edge of the tailgate LW are identified from the positional information of the first wing FW and the tailgate LW on the 3D mesh, respectively, and a straight line extending vertically is calculated connecting the leading edge of the first wing FW and the leading edge of the tailgate LW. The positional information of this straight line is then obtained as the leading edge FS of the cargo compartment 95. Similarly, the rear end of the first wing FW and the rear end of the tailgate LW are identified from the positional information of the first wing FW and the tailgate LW on the 3D mesh, respectively, and a straight line extending vertically is calculated connecting the rear end of the first wing FW and the rear end of the tailgate LW. The positional information of this straight line is then obtained as the rear end edge BS of the cargo compartment 95.

[0104] Furthermore, based on the positional information of the four sides consisting of the upper side US, lower side LS, front side FS, and rear side BS obtained in the above process, a cargo bed determination is performed to determine that the area enclosed by these four sides is inside the cargo compartment 95 (S8). The area determined to be inside the cargo compartment 95 through the above process is the one-side 1 / 2 area, or target area 95I.

[0105] As mentioned above, in Example 1, image information and depth information are repeatedly acquired at 60 frames per second (60 FPS). This acquisition of image information and depth information is repeated until the load amount is determined in S13 (Y in S13), and during this time, steps S3 to S10 are repeated.

[0106] After step S8, the target area 95I determined in S8 and the 3D mesh created in S5 are referenced to identify the cargo in the target area 95I (S9). Specifically, among the faces in the 3D mesh, those that are "within the target area 95I" and "facing the direction of the imaging element" are identified as cargo, and their volume is determined as the cargo volume. For reference, among the faces in the 3D mesh, those that are "within the target area 95I" and "facing the direction of the imaging element" correspond to box-shaped cargo or the sides of the pallet on which the cargo is placed. As previously mentioned, since the target area 95I is one-half of the area obtained based on the position information of the base of the first wing FW, the cargo identified in step S9 can be said to be cargo in the target area 95I. After step S9, the volume of the cargo in the target area 95I is calculated as the cargo volume (S10). The cargo volume calculated in step S10 is acquired as cargo volume data for one-half of the area (S11). At this time, naturally, the amount of cargo 97E located outside the target area 95E of the cargo compartment 95 and the amount of cargo 98 in the cargo compartment 96 of other trucks 91 are excluded from the cargo volume data.

[0107] Furthermore, the feasibility of the load quantity calculated in step S10 is determined (S13, S14). Specifically, the calculation of the position information of the upper edge US and the lower edge LS is attempted 120 times, and if the position information of the upper edge US and the lower edge LS can be obtained 60 or more times, the calculation is considered successful (Y in S12), and a final confirmation of load quantity registration is performed (S13). If the driver 99D, i.e., the operator, confirms that there are no problems, the load quantity is considered OK (Y in S13), and the process proceeds to the load quantity data registration process in S11 (S14), and the load quantity data for the one-sided 1 / 2 area is registered in the database (S15).

[0108] If the calculation of the load quantity calculated in step S11 is unsuccessful (N in S12), the process returns to step S3, and the acquisition of image information and depth information is repeated. If the load quantity is not OK (N in S13), the process returns to S3 and the acquisition of image information and depth information is repeated.

[0109] In Example 1, similar to the above, the driver 99D acquires image information and depth information from the other side of the cargo compartment 95 and registers the cargo volume data of the other half-area obtained by the same process as in steps S1 to S15 into the database. Then, based on the cargo volume data of one half-area and the cargo volume data of the other half-area, the driver determines the overall loading status of the cargo compartment 95 in the target truck 90. ​​Specifically, the driver calculates the amount of cargo that can be further loaded into the cargo compartment 95 of the target truck 90. ​​By feeding back the results, it becomes possible to suitably manage the number of routes and trips for cargo collection and delivery, as well as the cargo volume in each trip.

[0110] In the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, image data acquired by the imaging element 2 and depth information acquired by the sensor element 3 are used in combination to distinguish between the target area 95I and other areas. Therefore, according to the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, cargo in the target area 95I can be reliably identified, and the loading status of the cargo compartment 95 in the truck can be reliably grasped.

[0111] Furthermore, in the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, the imaging element 2 and sensor element 3 are the same ones mounted on the same mobile terminal 10. As a result, the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1 can reduce the costs required for the introduction and maintenance of the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system.

[0112] In the mobile cargo compartment condition measuring device 1 and cargo compartment condition measuring system of Example 1, their cooperation makes it possible to reliably grasp the loading conditions inside the cargo compartment 95 of a truck while keeping costs down.

[0113] (Example 2) The mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 are substantially the same as the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 1, but differ from the cargo compartment condition measuring system of Example 1 in the part of the application used for measuring cargo volume that is involved in measuring cargo volume. Figures 9 to 15 are schematic diagrams illustrating how the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 determine the loading conditions inside the cargo compartment of a truck.

[0114] For more details, Figure 9 schematically illustrates how image information and depth information related to the truck's cargo compartment are acquired, Figures 10 and 11 illustrate the operation of determining the four sides of the truck's cargo compartment in the image information, and Figures 12 to 15 illustrate the procedure for measuring cargo volume using the application used for cargo volume measurement.

[0115] The following describes the procedure for measuring cargo volume using the application used for cargo volume measurement in the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2.

[0116] The application includes multiple modes. The mobile device 10 is equipped with AR (augmented reality) functionality and an angular velocity sensor. Similar to Example 1, when the application for measuring load weight is launched, the load weight measurement screen opens and load weight measurement begins. As shown in Figure 12, the load weight list screen opens first. The load weight list screen displays a load weight measurement button and a back button. Operating the load weight measurement button allows you to select the imaging mode. Operating the back button returns you to the load weight list screen.

[0117] When selecting an imaging mode, the screen displays the following buttons: Record button, Split-screen button, Manual button, Auto button, and Back button. Of these, pressing the back button will return you to the load quantity list. Operating the split-image button will take you to the split-image mode, which will be described later. Pressing the MANUAL button will take you to the MANUAL IMAGING mode, which will be described later. Additionally, pressing the AUTO button will take you to the AUTO imaging mode, which will be described later.

[0118] When the above imaging mode is selected, pressing the record button will switch to video recording mode, and video capture and recording of the acquired video will begin. At this point, a "Scan Complete" button will appear on the screen. By pressing this button, the video capture and recording will end, the 3D model described above will be generated, and then the markers will become manually placeable.

[0119] As shown in Figure 10, in video recording mode, a target, roughly indicated by a cross shape, is displayed in the center of the 3D model shown on the screen.

[0120] When the mobile device 10 is tilted in this state, the angular velocity sensor detects the tilt of the mobile device 10, and the viewpoint of the 3D model displayed on the mobile device 10 changes according to that tilt.

[0121] By tilting the mobile device and aligning the target with one of the four corners of the cargo area in the 3D model on the screen, as shown in Figure 11, and then operating the position change button, the position of the target is selected as one of the four corners and a marker is placed at the selected position. By performing this operation sequentially, the positions of the four corners of the cargo bed are determined. Then, the cargo area is determined based on the marker, and the loading status of the entire cargo area is determined using the same process as in Example 1.

[0122] As mentioned above, the operation to determine the four corners of the cargo area can be performed in real time immediately after acquiring the image information of the cargo area, but it can also be done using, for example, a recorded 3D model of the cargo area. In this case, the operator who takes the images and the operator who determines the four corners of the cargo area may be different people. Furthermore, instead of the 3D model of the area around the truck mentioned above, either operator may manually determine the four corners of the cargo area based on a 2D model of the area around the truck.

[0123] In either case, by manually determining the four corners of the cargo area by the operator, it is possible to accurately determine the four corners of the cargo area even when, for example, the shooting environment of the cargo area is not suitable and it is difficult to automatically determine the four corners with high accuracy. This allows for reliable differentiation between cargo in the target area and cargo in other areas, thereby enabling a reliable understanding of the truck's loading status.

[0124] This section explains what happens when you select an imaging mode and then switch to the split imaging mode by operating the split imaging button. As shown in Figure 13, in split imaging mode, the record button, image capture button, manual button, auto button, and back button are displayed on the screen. Of these, pressing the back button will return you to the load quantity list. Pressing the image capture button allows you to select the image capture modes described above. Pressing the MANUAL button will take you to the MANUAL IMAGING mode, which will be described later. Additionally, pressing the AUTO button will take you to the AUTO imaging mode, which will be described later.

[0125] In the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2, it is possible to take multiple images of the truck's cargo compartment and acquire image information and depth information about the truck's cargo compartment based on the multiple images obtained.

[0126] More specifically, when the recording button is pressed on the split-image capture screen shown in Figure 13, the split-image capture recording screen opens, and the split-image capture of the video and the recording of each obtained video begin. At this time, a scan complete button is displayed on the screen, and pressing this scan complete button ends the split-image capture of still images and their recording.

[0127] As shown in Figure 9, in the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2, a 3D model of the area around the truck, including the cargo compartment, is generated by combining six images, namely images (1) to (6), which are obtained by segmenting the cargo compartment of the truck, using the calculation element 4. Each of the images (1) through (6) can be considered a part of the image information. Furthermore, since each of the images (1) through (6) is associated with corresponding depth information, it can be said that the 3D model around the truck includes both image information and depth information.

[0128] Furthermore, by combining multiple images as in Example 2 to generate a 3D model of the area around the truck, there is an advantage in that, for example, even when the distance from the cargo area to the imaging position is short and it is difficult to simultaneously image the entire cargo area, image information and depth information of the entire cargo area can be easily obtained. In addition, by dividing the image of the area around the truck into multiple images and acquiring them, the amount of data for each image is reduced, which also has the advantage of making it easier to send and receive the image data via communication.

[0129] In Example 2, a 3D model of the area around the truck was obtained by combining multiple videos, but a 3D model of the area around the truck may also be obtained by combining multiple still images. Furthermore, a 2D model of the area around the truck may be obtained instead of the 3D model. These 2D or 3D models only need to include image information and depth information.

[0130] Once the 3D model of the area around the track described above is generated, markers can be manually placed. Manual placement of markers is the same as in the video recording mode described above. At this time, in addition to the record button, the capture button, MANUAL button, AUTO button, and back button will be displayed on the screen. The capture button, MANUAL button, AUTO button, and back button are the same as in the video recording mode described above. By placing markers at the four corners of the cargo area, the overall loading status of the cargo area is determined, similar to the video recording mode.

[0131] This section explains what happens when you select an imaging mode and operate the MANUAL button to enter MANUAL imaging mode. As shown in Figure 14, in MANUAL imaging mode, imaging can be performed manually by the operator.

[0132] In MANUAL imaging mode, unlike the video recording mode described above, the 3D model around the track is displayed on the screen as if it were 2D.

[0133] In MANUAL imaging mode, the screen displays a recording button, as well as an image capture button, a split-screen button, an AUTO button, and a back button. The image capture button, split-screen button, AUTO button, and back button are the same as those described in the video recording mode.

[0134] In manual imaging mode, pressing the record button on the screen enables manual imaging, and video capture and recording of the acquired video begin. After capturing the image, pressing the "Scan Complete" button on the screen will end the video capture and recording process, and then you will be able to manually place the markers. The marker placement is as previously described.

[0135] This section explains what happens when you select an imaging mode and operate the AUTO button to enter AUTO imaging mode. As shown in Figure 15, when you enter AUTO imaging mode, the cargo area is automatically imaged, markers are automatically placed, and the overall loading status of the cargo area is also automatically determined. The image acquisition completion screen displays the image acquisition button, split image acquisition button, manual button, and back button. These buttons are as previously described.

[0136] The mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2, like the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 1, also determines the loading status inside the cargo compartment based on image information and depth information. Therefore, the mobile cargo compartment condition measuring device and cargo compartment condition measuring system of Example 2 can also reliably distinguish between cargo in the target area and cargo in other areas, thereby enabling a reliable understanding of the loading status of the truck.

[0137] Although the present invention has been described above, it is not limited to the embodiments described above, and it is possible to implement the invention by appropriately extracting and combining the elements described in the embodiments, and to make various modifications without departing from the spirit of the present invention. Furthermore, the specification of this invention discloses not only the reference relationships of each claim as originally filed, but also a technical concept that appropriately combines the matters described in each claim. [Explanation of Symbols]

[0138] 1: Mobile cargo compartment condition measuring device 10: Mobile devices 2: Imaging element 3: Sensor element 4: Calculation elements 9, 90: Truck 95: Cargo area US: Top edge LS: Bottom edge FS: Tip edge BS: Back edge LW: Aoribu FW: First Wing SW: 2nd Wing

Claims

1. A mobile terminal comprising an imaging element for acquiring image information, a sensor element for emitting electromagnetic waves and acquiring depth information from the electromagnetic waves reflected by the object, and a calculation element connected to the imaging element and the sensor element, The aforementioned imaging element acquires image information of the truck's cargo area from one side in the vehicle width direction. The sensor element acquires the depth information of the cargo compartment from one side to the other side. The aforementioned calculation element is Based on the aforementioned image information and depth information, a 3D mesh including positional information of the object is created on the image information. In the information of the aforementioned 3D mesh, The positions of the four sides of the cargo compartment, consisting of the top edge, bottom edge, front edge, and rear edge, on one side of the cargo compartment are obtained. The area enclosed by the four sides is determined to be inside the cargo compartment. Based on the above determination, the target area within the cargo compartment to be used for calculating the amount of cargo and / or empty space is identified, A surface located within the aforementioned target area and facing the imaging element is identified as the cargo, A mobile cargo compartment condition measuring device for determining the loading status of the cargo inside the cargo compartment on one side.

2. A cargo compartment condition measurement system for measuring the loading conditions inside the cargo compartment of a truck, using a terminal having an imaging element that acquires image information, a sensor element that emits electromagnetic waves and acquires depth information from electromagnetic waves reflected from an object, and a calculation element connected to the imaging element and the sensor element, The imaging element acquires the image information obtained by imaging the cargo area from one side in the vehicle width direction. The sensor element acquires the depth information of the cargo compartment from one side to the other. The aforementioned calculation element, Based on the aforementioned image information and depth information, a 3D mesh including positional information of the object is created on the image information. In the information of the aforementioned 3D mesh, The positions of the four sides of the cargo compartment, consisting of the top edge, bottom edge, front edge, and rear edge on one side, are obtained, and the area enclosed by these four sides is determined to be inside the cargo compartment. Based on the above determination, the target area within the cargo compartment to be used for calculating the amount of cargo and / or empty space is identified, A surface located within the aforementioned target area and facing the imaging element is identified as the cargo, A cargo compartment condition measurement system for determining the loading status of the cargo inside the cargo compartment on one side.

3. The cargo compartment condition measurement system according to claim 2, wherein the terminal is a mobile terminal.

4. The cargo compartment condition measurement system according to claim 2, wherein the calculation element scans the 3D mesh from top to bottom to detect horizontally extending edges on the 3D mesh, and based on this, the ceiling portion of the truck identified is obtained as the upper edge of the cargo compartment.

5. The cargo compartment condition measurement system according to claim 2 or 3, wherein the calculation element scans the 3D mesh from bottom to top to detect edges extending horizontally on the 3D mesh, and based on this, the tailgate portion of the truck identified is obtained as the lower edge of the cargo compartment.

6. The truck is a wing truck having a first wing that can be opened and closed to cover one side of the cargo compartment and a second wing that can be opened and closed to cover the other side of the cargo compartment. The aforementioned calculation element is The area located on one side in the vehicle width direction within the cargo compartment is identified as the target area. A cargo compartment condition measuring system according to claim 2 or claim 3, wherein the loading condition in the area of ​​the cargo compartment that is openably covered by the first wing is determined as the loading condition inside the cargo compartment.

Images