Detection system

Abstract

To provide a detection system that can detect a wide variety of equipment without increasing a cost.SOLUTION: A detection system includes: a depth sensor that is placed above equipment used for transporting and storing luggage and photographs a depth image that shows a distance to the equipment; a first image generation unit that extracts a depth value of a top surface of the equipment from the depth image and generates a binary image on the basis of the extracted depth value; a second image generation unit that generates a frame-shaped image having size of the top surface of the equipment in the depth image; and a determination unit that determines presence, orientation, and an area of the equipment on the basis of an overlap area where the binary image and the frame-shaped image overlap.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to detection systems. [Background technology]

[0002] One method for detecting a specific rectangular object of known size from an image is to determine that the object is a rectangular object by detecting a rectangle on the top surface of the object, then measure the lengths of the three sides of the object using a depth sensor or the like, and determine whether the measured lengths of the three sides match the known size.

[0003] Here, as a method for detecting a quadrangle from an image, a technique that detects contours and approximates them into a polygon is often used.

[0004] Furthermore, a device and a program have been proposed that use a depth sensor to estimate the size of a rectangular parallelepiped object (see, for example, Patent Document 1).

[0005] In addition, methods that use appearance information instead of size include a method of extracting and detecting the color of a target rectangular object from an RGB image of the target rectangular object, and a method of detecting objects in an image by having a deep learning model learn the object's appearance. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-081431 Summary of the Invention [Problem to be solved by the invention]

[0007] Incidentally, one example of equipment used for transporting and storing cargo is a container with an open top. In such containers, the contour of the container is hidden by long cargo placed inside (see Fig. 3A), and some containers have open sides (see Fig. 3B), making it difficult to detect the container using contour information from an image taken from above, as is the case with rectangular parallelepiped objects.

[0008] Furthermore, methods that extract color have problems such as the color changing depending on how the light hits the container, the possibility of rust, discoloration, and peeling paint due to aging, and false detection of the colors of surrounding objects, making it difficult to detect containers.

[0009] Furthermore, the method of detecting containers using a deep learning model requires a large number of object appearance images for learning and a high-performance PC for performing deep learning, which poses the problem of high costs.

[0010] An object of the present disclosure is to provide a detection system capable of detecting a wide variety of equipment without increasing costs. [Means for solving the problem]

[0011] In order to achieve the above object, the detection system of the present disclosure comprises: A depth sensor is disposed above a device used for transporting or storing luggage, and captures a depth image representing a distance to the device; a first image generating unit that extracts depth values ​​of the top surface of the equipment from the depth image and generates a binarized image based on the extracted depth values; a second image generating unit that generates a frame-shaped image having a size of the top surface of the equipment in the depth image; a determination unit that determines the presence, orientation, and area of ​​the device based on an overlap area where the binary image and the frame image overlap; Equipped with. Effect of the Invention

[0012] According to the present disclosure, a wide variety of equipment can be detected without increasing costs. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram illustrating a functional configuration of a detection system according to an embodiment of the present disclosure. [Diagram 2] 1 is a diagram illustrating a schematic diagram of a vehicle equipped with a detection system according to an embodiment of the present disclosure. [Figure 3A] FIG. 3A shows a container with a long item in it. [Figure 3B] FIG. 3B is a diagram showing an example of a container. [Figure 4] FIG. 4 is a diagram showing a landscape frame image displayed within a binarized image. [Diagram 5] FIG. 5 is a diagram showing a vertically oriented frame-shaped image displayed within a binarized image. [Figure 6] FIG. 6 is a diagram showing an example of a lattice window. [Figure 7] FIG. 7 is a flowchart showing an example of robot transport. [Figure 8] FIG. 8 is a flowchart showing an example of the operation of the detection system in this embodiment. [Figure 9] FIG. 9 is an image diagram of the measured data. [Figure 10] FIG. 10 is a flowchart showing an example of calculation of the accumulation rate. [Figure 11] FIG. 11 is a flowchart showing an example of processing by the overhead traveling vehicle. [Figure 12] FIG. 12 is a flowchart showing another example of the process of the overhead traveling vehicle. [Figure 13A] FIG. 13A is a front view of a horizontal axis fixed type depth sensor as viewed from the vehicle front-rear direction. [Figure 13B] FIG. 13B is a side view of the horizontal axis fixed type depth sensor as viewed from the vehicle width direction. [Figure 14A]FIG. 14A is a front view of the horizontal axis moving depth sensor as viewed from the vehicle front-rear direction. [Figure 14B] FIG. 14B is a side view of the lateral axis moving depth sensor as viewed from the vehicle width direction. [Figure 15A] FIG. 15A is a front view of the swing-type depth sensor as viewed from the vehicle front-rear direction. [Figure 15B] FIG. 15B is a side view of the oscillating depth sensor as viewed from the vehicle width direction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic functional configuration of a detection system according to an embodiment of the present disclosure. In FIG. 1, arrows indicate main data flows, and there may be data flows not shown in FIG. 1. In FIG. 1, each functional block shows a functional configuration, not a hardware (device) configuration. Therefore, the functional blocks shown in FIG. 1 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between functional blocks via any means, such as a data bus or a communication network.

[0015] FIG. 2 is a diagram showing a vehicle 1 equipped with a detection system according to an embodiment of the present disclosure. The detection system 100 according to the present embodiment is a device capable of detecting a wide variety of equipment used, for example, for transporting and storing luggage. In the following description, a container 5 (see FIG. 3B) installed in a luggage compartment of a truck (truck) for transporting luggage will be described as an example of the "equipment" according to the present embodiment. The container 5 according to the present embodiment is automatically transported in the luggage compartment. An automatic transport device 200 (see FIG. 1) for automatically transporting a container is a transport robot (hereinafter referred to as "robot 200") that enters a gap between a container 5 temporarily placed near an entrance / exit of the luggage compartment and a loading platform (floor surface of the luggage compartment), lifts the container 5, and transports the lifted container 5 to a predetermined position in the luggage compartment. The detection system 100 according to the present embodiment determines, for example, the presence, orientation (horizontal, vertical) and area (position) of the container 5 in the luggage compartment, and instructs the robot 200 to grasp the container according to the determination result.

[0016] The detection system 100 includes a depth sensor 2, a control device 3, and a memory unit 4.

[0017] (Depth Sensor 2) The depth sensor 2 is placed at a position above the container 5 (e.g., on the ceiling) which is temporarily placed near the entrance / exit of the cargo space, and photographs the container 5 from above. The depth sensor 2 collects light data (depth data) reflected from objects. Since the objects include the container 5, the depth data includes data indicating the distance to the container 5. The depth sensor 2 in this embodiment is placed on the ceiling of the cargo space, but when photographing a container placed on the bed of a flatbed truck without a cargo space from above, the depth sensor may be placed, for example, on the ceiling of the truck loading section of a warehouse or the like.

[0018] The appearance of the container 5 in this embodiment is diverse. FIG. 3B is a perspective view showing an example of a container. The container 5 has a predetermined size (width, depth, height). As an example of the container 5, there is one in which the upper surface portion 5a is configured as a substantially U-shaped frame body having three frames arranged at positions on one side in the width direction, the other side in the width direction, and the back side, and has an opening on the front side. The container 5 shown in FIG. 3B is called a cage dolly. In addition, as another example of the container 5, there is one in which the upper surface portion is configured as a rectangular frame body by four frames arranged at positions on one side in the width direction, the other side in the width direction, the front side, and the back side. In the following description, an example of a container 5 having an upper surface portion 5b (see FIG. 2) configured as a rectangular frame body will be described.

[0019] (Storage unit 4) The storage unit 4 is a storage device such as a ROM (Read Only Memory) that stores a BIOS (Basic Input Output System) of a computer that realizes the control device 3, a RAM (Random Access Memory) that serves as a working area for the control device 3, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores an OS (Operating System), application programs, various information referenced when the application programs are executed, and various maps. The control device 3 is installed in the vehicle and is configured as an integral part or separate part of one or more other on-board ECUs (Electronic Control Units) that control devices and systems equipped in the vehicle. On the other hand, the control device 3 may be installed outside the vehicle and connected to the depth sensor 2 and the robot 200 via a communication network.

[0020] The storage unit 4 stores in advance the width and depth on the depth image acquired by the depth sensor 2, and the height in real space.

[0021] (Control unit 30) The control unit 30 is a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the control device 3, and functions as an acquisition unit 31, a first image generation unit 32, a second image generation unit 33, a calculation unit 34, and a determination unit 35 by executing a program stored in the storage unit 4. Note that FIG. 1 shows an example in which the control device 3 is configured as a single device. However, the control device 3 may be realized by, for example, multiple processors, memory, and other computational resources. In this case, each unit constituting the control unit 30 is realized by at least one of multiple different processors executing a program.

[0022] (Acquisition part 31) The acquisition unit 31 acquires the depth data output from the depth sensor 2. The depth data is stored in the storage unit 4. As described above, the depth data includes data indicating the distance to the container 5.

[0023] (First image generating unit 32) The first image generating unit 32 extracts the depth value of the upper surface 5b of the container 5 based on the depth data read from the storage unit 4 and the height of the container 5 in the real space, and generates a binary image based on the extracted depth value. The binary image is an image generated based on the light data reflected from the upper surface 5b of the container 5, and is a binary image after depth extraction is performed using the depth value of the upper surface 5b of the container 5. FIG. 2 shows the container 5 placed on the installation surface, with the depth extraction range indicated by hatching, and the area within the depth extraction range in the binary image indicated by white.

[0024] (Second image generating unit 33) The detection process of the container 5 as an example will be described with reference to FIG. 4 and FIG. 5. FIG. 4 is a diagram showing a horizontal frame-shaped image displayed in a binary image. FIG. 5 is a diagram showing a vertical frame-shaped image displayed in a binary image. The second image generating unit 33 generates horizontal and vertical frame-shaped images having the size of the top surface 5b of the container 5 based on the width and depth of the container 5 on the image read from the storage unit 4. The second image generating unit 33 also generates a frame image according to the size of the binary image. Each of the generated horizontal and vertical frame-shaped images is stored in the storage unit 4. The orientation of the frame image (horizontal, vertical) can be selected by the user.

[0025] (Calculation unit 34) The control unit 30 slides the frame-shaped image within the binary image. In Fig. 4 and Fig. 5, the sliding path (path of the center point of the frame-shaped image) when the frame-shaped image is slid within the binary image is indicated by a thick arrow. When the frame-shaped image in the selected direction is slid within the binary image, the calculation unit 34 calculates the area of ​​the overlapping region where the binary image and the frame-shaped image overlap, and calculates the ratio of the area of ​​the overlapping region to the area of ​​the frame-shaped image. The calculation unit 34 calculates the ratio of the area of ​​the overlapping region every time the frame-shaped image is slid by a predetermined amount. Note that the predetermined amount is predetermined and adjustable. The calculation unit 34 calculates a score value when the ratio of the area of ​​the overlapping region is the largest while the frame-shaped image is slid by a predetermined amount. The score value is indicated in a plurality of stages. For example, the ratio when the score value is the lowest stage is 0% (a state in which the binary image and the frame-shaped image do not overlap at all). The ratio when the score value is the highest stage is 100% (a state in which the binary image and the frame-shaped image completely overlap).

[0026] (Judgment section 35) The determination unit 35 determines the presence, orientation, and area of ​​the container 5 based on the overlapping area where the binary image and the frame image overlap. Specifically, when the score value is equal to or greater than a predetermined value, the determination unit 35 determines the presence, orientation, and area of ​​the container 5 based on the slide position of the frame image. Note that, for example, when the score value when a horizontal frame image is used is less than a predetermined value, the determination unit 35 uses a vertical frame image to determine the presence, orientation, and area of ​​the container 5. Also, when the score value when either the horizontal or vertical frame image is used is less than a predetermined threshold value, the determination unit 35 determines the absence of the container 5. The predetermined value is stored in advance in the storage unit 4. Note that the control device 3 changes the predetermined value depending on conditions (for example, the frame width of the container). Also, the predetermined value can be changed by the terminal operation of the loading and unloading worker.

[0027] In addition, the determination unit 35 may determine a non-subject photographing area in the binarized image in which a non-subject object other than the container 5 (e.g., a wall in the luggage compartment) is photographed. This may result in a decrease in the detection accuracy of the container 5. As a method for preventing a decrease in the detection accuracy of the container 5, for example, when the depth sensor 2 is fixed in the luggage compartment, it is preferable to mask the non-subject photographing area. The mask excludes the non-subject photographing area from the sliding range in which the frame-shaped image is slid. In other words, the mask does not include the non-subject photographing area in the calculation area of ​​the score value.

[0028] In addition, in the above embodiment, a case has been described in which the entire container 5 is contained within the depth image. However, in cases where only a portion of the container 5 is reflected in the binary image, for example when the container 5 is large, the slide range can be expanded, for example by setting the slide start point of the frame-shaped image to the vertex of the image.

[0029] In the above embodiment, an example of sliding a frame-shaped image has been shown, but a lattice window in which multiple frame-shaped images are linked may be used as shown in Fig. 6. By using a lattice window, the sliding path can be shortened, and therefore the time required for detection can also be shortened.

[0030] Next, an example of robot transport in this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of robot transport in this embodiment. In this flow, a metal pallet (hereinafter, pallet) is used as the container 5, and pallet transport by a robot will be described in which the robot transports the pallet from a temporary placement position to a predetermined position in the luggage compartment. In addition, the number of pallets to be transported is assumed to be N.

[0031] First, in step S1, a pallet is temporarily placed near the entrance of the luggage compartment.

[0032] Next, in step S2, the interior of the luggage compartment is measured by the depth sensor 2. As a result, depth data is obtained.

[0033] Next, in step S3, the detection system 100 detects a pallet. Details of pallet detection will be described later as an example of the operation of the detection system 100.

[0034] Next, in step S4, the robot 200 performs pallet transport based on the detection result of the detection system 100.

[0035] Next, in step S5, it is determined whether or not N pallets have been transported. If N pallets have been transported (step S5: YES), this flow ends (loading complete). If N pallets have not been transported (step S5: NO), the process returns to before step S1.

[0036] Next, an example of the operation of the detection system 100 in this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the detection system 100 in this embodiment. This flow is started when a trigger for starting detection of the container 5 is turned on. Note that this flow may also be started when, for example, an entrance to the luggage compartment is opened.

[0037] First, in step S100, the acquisition unit 31 acquires depth data from the depth sensor 2.

[0038] Next, in step S110, the first image generating unit 32 binarizes the depth data using a threshold value of the depth extraction range, and generates a binarized image.

[0039] Next, in step S120, the calculation unit 34 slides the frame-shaped image within the binarized image and calculates a score value. Note that the frame-shaped image generated by the second image generation unit 33 is used.

[0040] Next, in step S130, the determination unit 35 determines, based on the score value, the presence, orientation, and area of ​​the container 5. After that, this flow ends.

[0041] The detection system 100 in the above embodiment is arranged above a container 5 used for transporting and storing luggage, etc., and includes a depth sensor 2 that captures depth data indicating the distance to the container 5, a first image generation unit 32 that extracts a depth value of the top surface 5a of the container from the depth data and generates a binary image based on the extracted depth value, a second image generation unit 33 that generates a frame-shaped image having a frame-like shape having the size of the top surface 5a of the container 5 in the binary image, and a determination unit 35 that determines the presence, orientation and area of ​​the container 5 based on the overlap area where the binary image and the frame-shaped image overlap.

[0042] With the above configuration, a frame-shaped image is generated to match the size of the top surface 5a of the container 5, and the presence of the container 5 is determined based on the generated frame-shaped image, making it possible to detect a wide variety of containers.

[0043] Furthermore, in the detection system 100 in the above embodiment, the determination unit determines the presence, orientation, and area of ​​the container 5 based on an overlapping area where a binary image and a frame-shaped image slid over the binary image overlap. Since the frame-shaped image is slid over the binary image and the presence, etc. of the container 5 is determined based on an overlapping area where the two overlap, it is possible to reliably detect the presence of a container whose size is known.

[0044] (Variation 1) The detection system 100 in the above embodiment determines the presence, orientation, and area of ​​the container 5, and instructs the robot 200 on the gripping position of the container based on the determination result. Based on the presence, orientation, and area of ​​the container 5, the robot 200 enters the gap between the container 5 and the loading platform (floor of the loading compartment), lifts up the container 5, and transports the lifted container 5 to a predetermined position in the loading compartment.

[0045] The control device 3 calculates the loading rate to grasp how effectively the cargo compartment is utilized. The loading rate is corrected so that 0% is empty and 100% is no more cargo can be loaded. The control device 3 transmits a transport instruction for the container 5 to the robot 200 based on the available space in the cargo compartment calculated from the depth data. Incidentally, when the container 5 is a cage cart (see FIG. 3B) or a metal pallet (also called a mesh pallet), the luggage placed in the cage cart or the like is a typical luggage that is prohibited from being loaded. The amount of such luggage loaded on each flight may vary from day to day. Therefore, the control device 3 in the first modification determines whether there is luggage that is prohibited from being loaded, and if there is luggage that is prohibited from being loaded, calculates the loading rate by assuming that there is luggage in the space above the luggage that is prohibited from being loaded.

[0046] In other words, the method of calculating the loading rate needs to be changed depending on whether or not there is any prohibited cargo. The loading rate calculation method will be described with reference to Fig. 9. Fig. 9 is an image diagram of measured data. First, the loading rate calculation method before the change will be described with reference to the vehicle depicted on the left side of the image shown in Fig. 9. In the vehicle depicted on the left side of Fig. 9, cargo that is prohibited from being loaded is indicated by diagonal hatching to the right. Using the load factor calculation method before the change, calculate the load factor (%) using the following formula (1). Loading rate (%) = ((L p ×W p ×H p ) + (other luggage)) x 100 / L v ×W v ×H v (1) Here, L v ,W v ,H v are the length, width, and height of the luggage compartment, and L p ,W p ,H p are the length, width, and height of the prohibited baggage dimensions.

[0047] Next, the changed loading rate calculation method will be described with reference to the vehicle depicted on the right side of the image shown in Fig. 9. In the vehicle depicted on the right side of Fig. 9, cargo that is prohibited from being loaded is shown by diagonal hatching to the right, and the space above the cargo that is prohibited from being loaded is shown by diagonal hatching to the left. Using the revised loading rate calculation method, the loading rate (%) is calculated using the following formula (2). Loading rate (%) = ((L p ×W p ×H v ) + (other luggage)) x 100 / L v ×W v ×H v (2) Here, L v ,W v ,H v are the length, width, and height of the luggage compartment, and L p ,W p ,H p are the length, width, and height of the prohibited baggage. v is H in the above formula (1). pThis dimension is the sum of the height of the space above any prohibited cargo.

[0048] In the detection system 100 in the first modification, the loading rate is calculated assuming that luggage is also loaded in the space above the luggage that is prohibited from being loaded, so that it is possible to accurately grasp how effectively the luggage compartment is being utilized.

[0049] (Variation 2) In the detection system 100 in the above embodiment, the depth sensor 2 is disposed at a position above (for example, on the ceiling) the container 5 that is temporarily placed near the entrance / exit of the luggage compartment. However, when the entrance to the luggage compartment is located at the rear end of the vehicle and the depth sensor 2 is located at an upper position near the entrance to the luggage compartment, the center of the luggage compartment in the fore-and-aft direction of the vehicle and the front end of the vehicle may be difficult to photograph from the depth sensor 2. This may reduce the accuracy of detecting the container 5 and increase the possibility of erroneously determining whether or not there is a prohibited cargo, which may reduce the accuracy of calculating the loading rate.

[0050] Therefore, an overhead traveling vehicle (not shown) that travels along the ceiling in the fore-and-aft direction of the vehicle is provided, and the depth sensor 2 is arranged on the overhead traveling vehicle. This prevents the center of the luggage compartment in the fore-and-aft direction of the vehicle and the front end of the vehicle from being difficult to photograph from the depth sensor 2, making it possible to accurately transport the container 5 and prevent a decrease in the accuracy of calculating the loading rate. The control device 3 drives and controls the overhead traveling vehicle, controls the photographing of the depth sensor 2, acquires depth data photographed by the depth sensor 2, detects prohibited overloading cargo based on the depth data, and calculates the loading rate based on the detection result of the prohibited overloading cargo.

[0051] Next, an example of calculation of an accumulation rate in this embodiment will be described with reference to Fig. 10. Fig. 10 is a flow chart showing an example of calculation of an accumulation rate in this embodiment. This flow starts after loading is completed.

[0052] First, in step S11, the detection system 100 confirms the completion of loading shown in step S5.

[0053] Next, in step S12, the interior of the luggage compartment is measured by the depth sensor 2. As a result, depth data is obtained.

[0054] Next, in step S13, the detection system 100 detects the pallet, thereby detecting any prohibited cargo.

[0055] Next, in step S14, the detection system 100 corrects the prohibited baggage.

[0056] Next, in step S15, the detection system 100 calculates the loading rate. Note that details of the calculation of the accumulation rate will be described later as an example of processing of an overhead traveling vehicle.

[0057] If the angle between the top of the container 5 (e.g., a cage cart) and the depth sensor 2 (camera) is acute, for example, if the depth sensor 2 and the container 5 are separated toward the rear of the vehicle, it becomes difficult to measure the top of the container 5, and the detection accuracy of the container 5 may decrease. Therefore, the depth sensor 2 may be attached to an overhead traveling vehicle installed on the ceiling.

[0058] Next, an example of the process of the overhead traveling vehicle will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the process of the overhead traveling vehicle. Here, the process when measuring the interior of the luggage compartment by the depth sensor 2 will be described.

[0059] First, in step S200, the control device 3 determines whether the measurement start trigger is on or off at regular intervals. If the measurement start trigger is on (step S200: YES), the process proceeds to step S210. If the measurement start trigger is off (step S200: NO), the process returns to before step S200.

[0060] In step S210, the control device 3 releases the power saving mode of the overhead traveling vehicle.

[0061] Next, in step S220, the control device 3 establishes communication with the robot 200.

[0062] Next, in step S230, the control device 3 controls the driving of the overhead traveling vehicle (not shown) and controls the imaging of the depth sensor 2 (driving and measuring).

[0063] Next, in step S240, the control device 3 stores the depth image captured by the depth sensor 2 in the storage unit 4 (creation and storage of depth data).

[0064] Next, in step S250, the control device 3 calculates the overloading prohibited points from the depth data by the detection system 100. Note that the overloading prohibited points may be calculated on the cloud side.

[0065] Next, in step S260, the control device 3 transmits data on the loading prohibited areas to the robot 200 based on the acquired depth data. When the robot 200 calculates the loading prohibited areas, it is not necessary to transmit the depth data and the loading prohibited areas. When the loading prohibited areas are calculated on the cloud side, the depth data and the loading prohibited areas are transmitted from the cloud side to the robot 200.

[0066] Next, in step S270, the control device 3 detects prohibited overloading baggage from the depth data and corrects the depth data (detects and corrects prohibited overloading baggage from the depth data).

[0067] Next, in step S280, the control device 3 calculates the loading rate from the corrected depth data and stores it in the memory unit 4.

[0068] Next, in step S290, the control device 3 turns on the power saving mode of the overhead traveling vehicle. This shuts down the main controller of the overhead traveling vehicle and keeps only functions such as waiting for triggers running. This saves power and makes it possible to obtain necessary data such as the loading rate.

[0069] Next, another example of the processing of the overhead traveling vehicle will be described with reference to Fig. 12. Fig. 12 is a flowchart showing another example of the processing of the overhead traveling vehicle. Here, as an example of the processing when the measurement results (depth data, loading rate) of the inside of the luggage compartment are transmitted to the cloud, the case where the measurement results are transmitted when the entrance of the luggage compartment is opened will be described.

[0070] First, in step S300, the control device 3 determines whether or not the opening of the entrance to the luggage compartment has been detected. If the opening of the door is detected (step S300: YES), the process proceeds to step S310. If the opening of the door is not detected (step S300: NO), the process returns to before step S300.

[0071] Next, in step S310, the control device 3 releases the power saving mode of the overhead traveling vehicle.

[0072] Next, in step S320, the control device 3 transmits data such as the depth data and the loading rate to the cloud.

[0073] Next, in step S330, the control device 3 turns on the power saving mode of the overhead traveling vehicle, which makes it possible to achieve power saving when transmitting necessary data such as the loading rate to the cloud.

[0074] (Variation 3) In the above-mentioned modified example 2, the overhead traveling vehicle (not shown) that travels in the fore-and-aft direction of the vehicle along the ceiling and the depth sensor 2 arranged on the overhead traveling vehicle have been described. Since the overhead traveling vehicle travels in the fore-and-aft direction of the vehicle in the center in the width direction of the vehicle, both ends in the vehicle width direction in the luggage compartment may be difficult to photograph from the depth sensor 2. This reduces the accuracy of detecting the container 5, increasing the possibility of erroneously determining whether or not there is a prohibited cargo, which may reduce the accuracy of calculating the loading rate.

[0075] Therefore, in Modification 3, multiple depth sensors 2 are arranged at a predetermined interval in the vehicle width direction (fixed horizontal axis type), the depth sensors 2 are configured to be movable in the vehicle width direction (movable horizontal axis type), or the depth sensors 2 are configured to be swivellable in the vehicle width direction (swivel type). This prevents both ends of the luggage compartment in the vehicle width direction from being difficult for the depth sensors 2 to photograph. Below, the fixed horizontal axis type depth sensor, the movable horizontal axis type depth sensor, and the swivel type depth sensor will be explained. Note that in Modification 3, the configurations different from the embodiment will be mainly explained, and the same configurations will be assigned the same numbers and their explanation will be omitted.

[0076] First, the horizontal shaft fixed type depth sensor will be described with reference to Fig. 13A and Fig. 13B. Fig. 13A is a front view of the horizontal shaft fixed type depth sensor when viewed from the vehicle longitudinal direction. Fig. 13B is a side view of the horizontal shaft fixed type depth sensor when viewed from the vehicle width direction. As shown in Fig. 13A and Fig. 13B, the horizontal shaft fixed type depth sensor has five monocular cameras 2a, a rail 6, an overhead traveling vehicle 7, and a horizontal shaft 8. The rail 6 is disposed on the ceiling of the luggage compartment and extends in the vehicle longitudinal direction. The overhead traveling vehicle 7 is capable of self-propelling in the vehicle longitudinal direction along the rail 6. The horizontal shaft 8 is attached to the overhead traveling vehicle 7 and extends in the vehicle width direction. Each of the five monocular cameras 2a is disposed at a position on the horizontal shaft 8 in the vehicle width direction center, both ends in the vehicle width direction, and between the center and both ends in the vehicle width direction. As a result, for example, the cameras 2a arranged at each of the two ends of the vehicle width direction can photograph both ends of the vehicle width direction inside the luggage compartment from above, so that both ends of the vehicle width direction inside the luggage compartment do not become locations that are difficult to photograph for the cameras 2a. This makes it possible to prevent a decrease in the accuracy of detecting the containers 5 and also to suppress a decrease in the accuracy of calculating the loading rate.

[0077] Next, the horizontal axis moving type depth sensor will be described with reference to Fig. 14A and Fig. 14B. Fig. 14A is a front view of the horizontal axis moving type depth sensor when viewed from the vehicle longitudinal direction. Fig. 14B is a side view of the horizontal axis moving type depth sensor when viewed from the vehicle transverse direction. As shown in Fig. 14A and Fig. 14B, the horizontal axis moving type depth sensor has one monocular camera 2a (or a stereo camera), a rail 6, an overhead traveling vehicle 7, a rack 9, and a guide 9a. The rail 6 is disposed on the ceiling of the luggage compartment and extends in the vehicle longitudinal direction. The overhead traveling vehicle 7 is self-propelled in the vehicle longitudinal direction along the rail 6 and has a rack driving gear 7a. The rack 9 is movable in the vehicle transverse direction by the rack driving gear 7a. The guide 9a is fixed to the overhead traveling vehicle 7 and extends in the vehicle transverse direction. The camera 2a is fixed to the rack 9 and can be guided by the guide 9a. When the rack 9 is moved in the vehicle width direction by the rack driving gear 7a, the camera 2a fixed to the rack 9 is guided by the guide 9a and becomes movable in the vehicle width direction. As a result, for example, the camera 2a moved to each position at both ends in the vehicle width direction can photograph both ends of the luggage compartment in the vehicle width direction from above, so that both ends of the luggage compartment in the vehicle width direction do not become places that are difficult to photograph for the camera 2a, making it possible to prevent a decrease in the accuracy of detecting the container 5 and also to suppress a decrease in the accuracy of calculating the loading rate.

[0078] Next, the oscillating depth sensor will be described with reference to Fig. 15A and Fig. 15B. Fig. 15A is a front view of the oscillating depth sensor as viewed from the vehicle longitudinal direction. Fig. 15B is a side view of the oscillating depth sensor as viewed from the vehicle transverse direction. As shown in Fig. 15A and Fig. 15B, the oscillating depth sensor has one monocular camera 2a (or a stereo camera), a rail 6, an overhead traveling vehicle 7, a rotating shaft 9b, and a driven gear 9c. The rail 6 is disposed on the ceiling of the luggage compartment and extends in the vehicle longitudinal direction. The overhead traveling vehicle 7 is self-propelled in the vehicle longitudinal direction along the rail 6, and has a driving gear 7b and a bearing portion 7c. The rotating shaft 9b is attached to the overhead traveling vehicle 7 via the bearing portion 7c and extends in the vehicle longitudinal direction. The rotating shaft 9b is rotatable around its axis by the bearing portion 7c. A belt is wound around the driven gear 9c and the driving gear 7b, and the driven gear 9c is fitted to and fixed to the rotating shaft 9b. The camera 2a is fixed to the rotating shaft 9b via the arm 2b. When the driven gear 9c is rotated by the driving gear 7b, the rotating shaft 9b and the arm 2b rotate integrally, which allows the camera a to swing in the vehicle width direction. As a result, for example, the optical axis, which is the center of the angle of view of the camera 2a, can be directed toward both ends of the vehicle width direction in the luggage compartment, so that both ends of the vehicle width direction in the luggage compartment are not difficult to photograph from the camera 2a, which makes it possible to prevent a decrease in the accuracy of detecting the container 5 and also to suppress a decrease in the accuracy of calculating the loading rate.

[0079] In addition, the above-mentioned embodiments are merely examples of the embodiment of the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by them. In other words, the present disclosure can be embodied in various forms without departing from the gist or main features thereof. [Industrial Applicability]

[0080] The present disclosure is preferably used in vehicles equipped with a detection system that is required to detect a wide variety of equipment without increasing costs. [Explanation of symbols]

[0081] 1 vehicle 2 Depth Sensor 2a Monocular camera 3. Control device 4 Storage section 5. Container 5a Top part 5b Top part 6 Rail 7 Overhead vehicle 7a Rack drive gear 7b Drive gear 7c Bearing part 8 Horizontal axis 9. Rack 9a Guide 9b Rotation axis 9c Driven Gear 30 Control section 31 Acquisition Department 32 First image generating unit 33 Second image generating unit 34 Calculation section 35 Judgment section 100 Detection System 200 Robots

Claims

1. A depth sensor is disposed above a device used for transporting or storing luggage, and captures a depth image representing a distance to the device; a first image generating unit that extracts a depth value of a top surface portion of the equipment from the depth image and generates a binarized image based on the extracted depth value; a second image generating unit that generates a frame-shaped image having a size of the top surface of the equipment in the depth image; a determination unit that determines the presence, orientation, and area of ​​the device based on an overlap area where the binary image and the frame image overlap; Equipped with Detection system.

2. the determination unit determines the presence, orientation, and area of ​​the equipment based on an overlapping area where the binary image and the frame image slid relative to the binary image overlap; The detection system of claim 1 .

Images