Detection System

Abstract

To provide a detection system capable of detecting a wide variety of equipment without increasing costs. [Solution] The detection system comprises a depth sensor that is placed above equipment used for transporting or storing luggage, etc., and captures a depth image indicating the distance to the equipment, a first image generation unit that extracts depth values ​​of the top surface of the equipment from the depth image and generates a binary image based on the extracted depth values, a second image generation unit that generates a frame-shaped image having the size of the top surface of the equipment in the depth image, and a determination unit that determines the presence, orientation and area of ​​the equipment based on the overlap area where the binary image and the frame-shaped image overlap.

Description

[Technical Field]

[0001] The present disclosure relates to detection systems. [Background technology]

[0002] One method for detecting a specific rectangular object of known size from an image is to determine that the object is a rectangular object by detecting a rectangle on the top surface of the rectangular object, then measure the lengths of the three sides of the object using a depth sensor or the like, and determine whether the measured lengths of the three sides match the known size.

[0003] Here, a commonly used method for detecting a quadrangle from an image is to detect a contour and approximate it into a polygon.

[0004] Furthermore, a device and a program have been proposed that use a depth sensor to estimate the size of a rectangular parallelepiped object (see, for example, Patent Document 1).

[0005] In addition, methods that use appearance information instead of size include extracting and detecting the color of the target rectangular object from an RGB image of the object, and detecting objects in an image by having a deep learning model learn the object's appearance. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-081431 Summary of the Invention [Problem to be solved by the invention]

[0007] Incidentally, containers with open tops are examples of equipment used for transporting and storing cargo. In such containers, the contours of the containers are often hidden by long cargo inside (see Figure 3A), and some containers have open sides (see Figure 3B), making it difficult to detect containers using contour information from an image taken from above, as is the case with rectangular parallelepiped objects.

[0008] Furthermore, methods that extract color have problems such as the color changing depending on how the light hits the container, the possibility of rust, discoloration, or paint peeling due to aging, and false detection of the color of surrounding objects, making it difficult to detect containers.

[0009] Furthermore, the method of detecting containers using a deep learning model requires a large number of object appearance images for learning and a high-performance PC for performing deep learning, which poses the problem of high costs.

[0010] An object of the present disclosure is to provide a detection system that is capable of detecting a wide variety of equipment without increasing costs. [Means for solving the problem]

[0011] In order to achieve the above object, the detection system in the present disclosure comprises: Carrying and securing luggage In the tube Used , open top It is placed above the equipment, From the upper position a depth sensor that captures a depth image that indicates the distance to the equipment; a first image generation unit that extracts depth values ​​of the upper surface of the equipment from the depth image and generates a binarized image based on the extracted depth values; The upper surface of the equipment in the depth image Based on width and depth length a second image generating unit that generates a frame-shaped image; While sliding the frame-shaped image within the binarized image, An overlapping region where the binary image and the frame-shaped image overlap. a calculation unit that calculates a score value that indicates the area ratio of If the calculated score value is equal to or greater than a predetermined value, the slide position of the frame-shaped image is a determination unit that determines the presence, orientation, and area of ​​the device based on the Equipped with. [Effects of the Invention]

[0012] According to the present disclosure, a wide variety of devices can be detected without increasing costs. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram schematically illustrating a functional configuration of a detection system according to an embodiment of the present disclosure. [Figure 2] 1 is a diagram schematically illustrating a vehicle equipped with a detection system according to an embodiment of the present disclosure. [Figure 3A] FIG. 3A shows a container containing a long item. [Figure 3B] FIG. 3B is a diagram showing an example of a container. [Figure 4] FIG. 4 is a diagram showing a landscape frame image displayed within a binarized image. [Figure 5] FIG. 5 is a diagram showing a vertically oriented frame-shaped image displayed within a binarized image. [Figure 6] FIG. 6 is a diagram showing an example of a lattice window. [Figure 7] FIG. 7 is a flowchart showing an example of robot transport. [Figure 8] FIG. 8 is a flowchart showing an example of the operation of the detection system according to this embodiment. [Figure 9] FIG. 9 is an image diagram of the measured data. [Figure 10] FIG. 10 is a flowchart showing an example of calculation of the accumulation rate. [Figure 11] FIG. 11 is a flowchart showing an example of processing by the overhead traveling vehicle. [Figure 12] FIG. 12 is a flowchart showing another example of the processing of the overhead traveling vehicle. [Figure 13A] FIG. 13A is a front view of a horizontal axis fixed type depth sensor as viewed from the vehicle longitudinal direction. [Figure 13B]FIG. 13B is a side view of the horizontal axis fixed type depth sensor as viewed from the vehicle width direction. [Figure 14A] FIG. 14A is a front view of the horizontal axis moving depth sensor as seen from the front-rear direction of the vehicle. [Figure 14B] FIG. 14B is a side view of the horizontal axis moving depth sensor as seen from the vehicle width direction. [Figure 15A] FIG. 15A is a front view of the swing-type depth sensor as seen from the front-rear direction of the vehicle. [Figure 15B] FIG. 15B is a side view of the swing-type depth sensor as viewed from the vehicle width direction. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a block diagram schematically illustrating the functional configuration of a detection system according to an embodiment of the present disclosure. In FIG. 1, arrows indicate major data flows, and there may be other data flows not shown in FIG. 1. In FIG. 1, each functional block indicates a functional unit configuration, rather than a hardware (device) unit configuration. Therefore, the functional blocks shown in FIG. 1 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between functional blocks via any means, such as a data bus or a communication network.

[0015] FIG. 2 is a diagram schematically illustrating a vehicle 1 equipped with a detection system according to an embodiment of the present disclosure. The detection system 100 according to the present embodiment is a device capable of detecting a wide variety of equipment used, for example, for transporting and storing luggage. In the following description, a container 5 (see FIG. 3B) installed in the luggage compartment of a truck (truck) for transporting luggage will be described as an example of the "equipment" according to the present embodiment. Note that the container 5 according to the present embodiment is automatically transported within the luggage compartment. An automatic transport device 200 (see FIG. 1) that automatically transports the container is a transport robot (hereinafter referred to as "robot 200") that enters a gap between the container 5 temporarily placed near the entrance / exit of the luggage compartment and the loading platform (floor of the luggage compartment), lifts the container 5, and transports the lifted container 5 to a predetermined position within the luggage compartment. The detection system 100 according to the present embodiment determines, for example, the presence, orientation (horizontal, vertical), and area (position) of the container 5 within the luggage compartment, and instructs the robot 200 on a position to grasp the container based on the determination result.

[0016] The detection system 100 includes a depth sensor 2, a control device 3, and a memory unit 4.

[0017] (Depth Sensor 2) The depth sensor 2 is placed above (for example, on the ceiling) a container 5 that is temporarily placed near the entrance / exit of the cargo space, and photographs the container 5 from above. The depth sensor 2 collects light data (depth data) reflected from objects. Because the objects include the container 5, the depth data includes data indicating the distance to the container 5. In this embodiment, the depth sensor 2 is placed on the ceiling of the cargo space, but when photographing a container placed on the bed of a flatbed truck without a cargo space from above, the depth sensor may be placed, for example, on the ceiling of the truck loading area of ​​a warehouse or the like.

[0018] The appearance of the container 5 in this embodiment varies widely. FIG. 3B is a perspective view showing an example of a container. The container 5 has a predetermined size (width, depth, and height). One example of the container 5 is one in which the upper surface 5a thereof is configured as a substantially U-shaped frame having three frames arranged at one widthwise side, the other widthwise side, and the rear end, and an opening on the front side. The container 5 shown in FIG. 3B is called a car dolly. Another example of the container 5 is one in which the upper surface thereof is configured as a rectangular frame having four frames arranged at one widthwise side, the other widthwise side, the front side, and the rear end. In the following explanation, an example of a container 5 having an upper surface 5b (see FIG. 2) configured as a rectangular frame will be described.

[0019] (Storage unit 4) The storage unit 4 is a storage device such as a ROM (Read Only Memory) that stores the BIOS (Basic Input Output System) of a computer that implements the control device 3, a RAM (Random Access Memory) that serves as a work area for the control device 3, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores the OS (Operating System), application programs, various information referenced when the application programs are executed, and various maps. The control device 3 is installed inside the vehicle and configured as an integral part of or separate from one or more other on-board ECUs (Electronic Control Units) that control devices and systems equipped on the vehicle. Alternatively, the control device 3 may be installed outside the vehicle and connected to the depth sensor 2 and the robot 200 via a communication network.

[0020] The storage unit 4 stores in advance the width and depth on the depth image acquired by the depth sensor 2 and the height in real space.

[0021] (control unit 30) The control unit 30 is a processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) of the control device 3, and functions as an acquisition unit 31, a first image generation unit 32, a second image generation unit 33, a calculation unit 34, and a determination unit 35 by executing programs stored in the storage unit 4. Note that FIG. 1 shows an example in which the control device 3 is configured as a single device. However, the control device 3 may be realized by, for example, multiple processors, memories, and other computational resources. In this case, each unit constituting the control unit 30 is realized by at least one of multiple different processors executing a program.

[0022] (Acquisition part 31) The acquisition unit 31 acquires the depth data output from the depth sensor 2. The depth data is stored in the storage unit 4. As described above, the depth data includes data indicating the distance to the container 5.

[0023] (First image generation unit 32) The first image generation unit 32 extracts depth values ​​of the upper surface 5b of the container 5 based on the depth data read from the storage unit 4 and the height of the container 5 in real space, and generates a binary image based on the extracted depth values. The binary image is an image generated based on data of light reflected from the upper surface 5b of the container 5, and is a binary image after depth extraction is performed using the depth values ​​of the upper surface 5b of the container 5. Figure 2 shows the container 5 placed on the installation surface, with the depth extraction range indicated by hatching and the area within the depth extraction range in the binary image indicated in white.

[0024] (Second image generation unit 33) An example of the detection process for a container 5 will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram showing a horizontally oriented frame image displayed within a binary image. FIG. 5 is a diagram showing a vertically oriented frame image displayed within the binary image. The second image generation unit 33 generates horizontally and vertically oriented frame images each having a size equal to the size of the top surface 5b of the container 5 based on the width and depth of the container 5 in the image read from the storage unit 4. The second image generation unit 33 also generates a frame image according to the size of the binary image. The generated horizontally and vertically oriented frame images are each stored in the storage unit 4. The orientation of the frame image (horizontally or vertically) can be selected by the user.

[0025] (Calculation unit 34) The control unit 30 slides the frame image within the binary image. In FIGS. 4 and 5, the sliding path (path of the center point of the frame image) when the frame image is slid within the binary image is indicated by a thick arrow. The calculation unit 34 calculates the area of ​​the overlapping region where the binary image and the frame image overlap when the frame image is slid in the selected direction within the binary image, and calculates the ratio of the area of ​​the overlapping region to the area of ​​the frame image. The calculation unit 34 calculates the ratio of the area of ​​the overlapping region each time the frame image is slid by a predetermined amount. Note that this predetermined amount is predetermined and adjustable. The calculation unit 34 calculates a score value when the ratio of the area of ​​the overlapping region is largest while the frame image is slid by the predetermined amount. The score value is expressed in multiple stages. For example, the lowest score level is 0% (a state in which the binary image and the frame image do not overlap at all). The highest score level is 100% (a state in which the binary image and the frame image completely overlap).

[0026] (Judgment section 35) The determination unit 35 determines the presence, orientation, and area of ​​the container 5 based on the overlap area where the binary image and the frame image overlap. Specifically, if the score value is equal to or greater than a predetermined value, the determination unit 35 determines the presence, orientation, and area of ​​the container 5 based on the sliding position of the frame image. Note that, for example, if the score value when a horizontally oriented frame image is used is less than the predetermined value, the determination unit 35 determines the presence, orientation, and area of ​​the container 5 using a vertically oriented frame image. Furthermore, if the score value when both the horizontally and vertically oriented frame images are used is less than the predetermined threshold, the determination unit 35 determines that the container 5 is not present. The predetermined value is pre-stored in the memory unit 4. Note that the control device 3 changes the predetermined value depending on conditions (e.g., the frame width of the container). The predetermined value can also be changed by the loading and unloading worker's terminal operation.

[0027] It should be noted that the determination unit 35 may determine a non-target object photographed area in the binarized image in which a non-target object other than the container 5 (for example, a wall in the luggage compartment) is photographed. This may result in a decrease in the accuracy of detecting the container 5. As a method for preventing a decrease in the accuracy of detecting the container 5, for example, when the depth sensor 2 is fixed inside the luggage compartment, it is preferable to mask the non-target object photographed area. The mask excludes the non-target object photographed area from the sliding range in which the frame-shaped image is slid. In other words, the mask excludes the non-target object photographed area from the calculation area of ​​the score value.

[0028] Furthermore, in the above-described embodiment, the case where the entire container 5 is contained within the depth image has been described. However, if only a portion of the container 5 is reflected in the binary image, for example, when the container 5 is large, the slide range can be expanded, for example, by setting the slide start point of the frame-shaped image to the vertex of the image.

[0029] In the above embodiment, an example of sliding a frame-shaped image is shown, but a lattice window in which multiple frame-shaped images are connected may also be used, as shown in Fig. 6. Using a lattice window can shorten the sliding path, thereby reducing the time required for detection.

[0030] Next, an example of robot transport in this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of robot transport in this embodiment. In this flow, a metal pallet (hereinafter referred to as pallet) is used as the container 5, and pallet transport by a robot will be described in which the robot transports the pallet from a temporary storage position to a predetermined position in the luggage compartment. In addition, the number of pallets to be transported is assumed to be N.

[0031] First, in step S1, a pallet is temporarily placed near the entrance of the luggage compartment.

[0032] Next, in step S2, the interior of the luggage compartment is measured by the depth sensor 2. As a result, depth data is obtained.

[0033] Next, in step S3, the detection system 100 detects the pallet. Details of pallet detection will be described later as an example of the operation of the detection system 100.

[0034] Next, in step S4, the robot 200 performs pallet transport based on the detection result of the detection system 100.

[0035] Next, in step S5, it is determined whether or not N pallets have been transported. If N pallets have been transported (step S5: YES), this flow ends (loading complete). If N pallets have not been transported to complete loading (step S5: NO), the process returns to before step S1.

[0036] Next, an example of the operation of the detection system 100 in this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the detection system 100 in this embodiment. This flow starts when a trigger for starting detection of the container 5 is turned on. Note that this flow may also start when, for example, an entrance to the luggage compartment is opened.

[0037] First, in step S100, the acquisition unit 31 acquires depth data from the depth sensor 2.

[0038] Next, in step S110, the first image generating unit 32 binarizes the depth data using the threshold value of the depth extraction range to generate a binarized image.

[0039] Next, in step S120, the calculation unit 34 slides the frame-shaped image within the binarized image and calculates the score value. Note that the frame-shaped image used is the one generated by the second image generation unit 33.

[0040] Next, in step S130, the determination unit 35 determines, based on the score value, the presence, orientation, and area of ​​the container 5. After that, this flow ends.

[0041] The detection system 100 in the above embodiment includes a depth sensor 2 that is placed above a container 5 used for transporting and storing cargo, and captures depth data representing the distance to the container 5; a first image generation unit 32 that extracts depth values ​​of the top surface 5a of the container from the depth data and generates a binary image based on the extracted depth values; a second image generation unit 33 that generates a frame-shaped image having a frame-like shape having the size of the top surface 5a of the container 5 in the binary image; and a determination unit 35 that determines the presence, orientation, and area of ​​the container 5 based on the overlap area where the binary image and the frame-shaped image overlap.

[0042] With the above configuration, a frame-shaped image is generated to match the size of the top surface 5a of the container 5, and the presence of the container 5 can be determined based on the generated frame-shaped image, making it possible to detect a wide variety of containers.

[0043] Furthermore, in the detection system 100 according to the above embodiment, the determination unit determines the presence, orientation, and area of ​​the container 5 based on the overlapping area where the binary image and the frame-shaped image slid over the binary image overlap. Since the frame-shaped image is slid over the binary image and the presence, etc., of the container 5 is determined based on the overlapping area where the two images overlap, it is possible to reliably detect the presence of a container whose size is known.

[0044] (Variation 1) The detection system 100 in the above embodiment determines the presence, orientation, and area of ​​the container 5, and instructs the robot 200 on the position to grasp the container based on the determination result. Based on the presence, orientation, and area of ​​the container 5, the robot 200 enters the gap between the container 5 and the loading platform (floor of the cargo compartment), lifts up the container 5, and transports the lifted container 5 to a predetermined position in the cargo compartment.

[0045] The control device 3 calculates the loading rate to grasp how effectively the cargo compartment is being used. The loading rate is corrected so that 0% indicates an empty load, and 100% indicates that no more cargo can be loaded. The control device 3 sends a transport instruction for the container 5 to the robot 200 based on the available space in the cargo compartment calculated from the depth data. Incidentally, when the container 5 is a cage cart (see FIG. 3B) or a metal pallet (also called a mesh pallet), the cargo placed in the cage cart or the like is a typical cargo that is prohibited from being overloaded. The amount of such cargo loaded on each flight may vary from day to day. Therefore, the control device 3 in Modification 1 determines whether or not there is any cargo that is prohibited from being overloaded, and if there is any cargo that is prohibited from being overloaded, calculates the loading rate by assuming that there is cargo in the space above the cargo that is prohibited from being overloaded.

[0046] In other words, the method of calculating the loading rate needs to be changed depending on whether or not there is any cargo that is prohibited from being loaded. The loading rate calculation method will be described with reference to Fig. 9. Fig. 9 is an image diagram of measured data. First, the loading rate calculation method before the change will be described with reference to the vehicle depicted on the left side of the image shown in Fig. 9. In the vehicle depicted on the left side of Fig. 9, cargo that cannot be loaded is indicated by diagonal hatching to the right. Using the loading rate calculation method before the change, the loading rate (%) is calculated using the following formula (1). Loading rate (%) = ((L p ×W p ×H p ) + (other luggage)) x 100 / L v ×W v ×H v ···(1) where L v ,W v ,H v are the length, width, and height of the luggage compartment, and L p ,W p ,H p are the length, width, and height of the prohibited cargo dimensions.

[0047] Next, the modified loading rate calculation method will be described with reference to the vehicle depicted on the right side of the image shown in Fig. 9. In the vehicle depicted on the right side of Fig. 9, cargo prohibited from being loaded is indicated by diagonal hatching to the right, and the space above the cargo prohibited from being loaded is indicated by diagonal hatching to the left. Using the revised loading rate calculation method, the loading rate (%) is calculated using the following formula (2). Loading rate (%) = ((L p ×W p ×H v ) + (other luggage)) x 100 / L v ×W v ×H v ···(2) where L v ,W v ,H v are the length, width, and height of the luggage compartment, and L p ,W p ,H p are the length, width, and height of the prohibited cargo dimensions. v is H in the above equation (1). pThis dimension is the height of the space above any cargo that cannot be loaded on top of the cargo.

[0048] In the detection system 100 in the first modification, the loading rate is calculated assuming that luggage is also loaded in the space above the luggage that is prohibited from being loaded, so it is possible to accurately grasp how effectively the luggage compartment is being used.

[0049] (Variation 2) In the detection system 100 according to the above embodiment, the depth sensor 2 is disposed above (for example, on the ceiling) the container 5 that is temporarily placed near the entrance of the luggage compartment. However, if the entrance to the luggage compartment is located at the rear end of the vehicle and the depth sensor 2 is located at an upper position near the entrance to the luggage compartment, the center of the luggage compartment in the fore-and-aft direction of the vehicle and the front end of the vehicle may be difficult to photograph from the depth sensor 2. This may reduce the accuracy of detecting the container 5 and increase the possibility of erroneously determining whether or not there is any prohibited cargo, which may reduce the accuracy of calculating the loading ratio.

[0050] Therefore, an overhead traveling vehicle (not shown) that travels along the ceiling in the fore-and-aft direction of the vehicle is provided, and a depth sensor 2 is arranged on the overhead traveling vehicle. This prevents the center of the luggage compartment in the fore-and-aft direction of the vehicle and the front end of the vehicle from being difficult to photograph from the depth sensor 2, making it possible to accurately transport the container 5 and prevent a decrease in the accuracy of calculating the loading rate. The control device 3 controls the driving of the overhead traveling vehicle, controls the photography of the depth sensor 2, acquires depth data photographed by the depth sensor 2, detects prohibited overloading cargo based on the depth data, and calculates the loading rate based on the detection result of prohibited overloading cargo.

[0051] Next, an example of calculation of the accumulation rate in this embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of calculation of the accumulation rate in this embodiment. This flow starts after loading is completed.

[0052] First, in step S11, the detection system 100 confirms the completion of loading shown in step S5.

[0053] Next, in step S12, the interior of the luggage compartment is measured by the depth sensor 2. As a result, depth data is obtained.

[0054] Next, in step S13, the detection system 100 detects the pallet, thereby detecting the prohibited overloading baggage.

[0055] Next, in step S14, the detection system 100 corrects the prohibited overload baggage.

[0056] Next, in step S15, the detection system 100 calculates the loading rate. Details of the calculation of the loading rate will be described later as an example of processing for an overhead traveling vehicle.

[0057] If the angle formed between the top of the container 5 (e.g., a cart) and the depth sensor 2 (camera) is acute, for example, if the depth sensor 2 and the container 5 are separated toward the rear of the vehicle, it may be difficult to measure the top of the container 5, and the detection accuracy of the container 5 may decrease. Therefore, the depth sensor 2 may be attached to an overhead traveling vehicle installed on the ceiling.

[0058] Next, an example of processing by the overhead traveling vehicle will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of processing by the overhead traveling vehicle. Here, processing when measuring the interior of the luggage compartment using the depth sensor 2 will be described.

[0059] First, in step S200, the control device 3 determines whether the measurement start trigger is on or off at regular intervals. If the measurement start trigger is on (step S200: YES), the process proceeds to step S210. If the measurement start trigger is off (step S200: NO), the process returns to before step S200.

[0060] In step S210, the control device 3 cancels the power saving mode of the overhead traveling vehicle.

[0061] Next, in step S220, the control device 3 establishes communication with the robot 200.

[0062] Next, in step S230, the control device 3 controls the driving of the overhead traveling vehicle (not shown) and controls the photography of the depth sensor 2 (driving and measuring).

[0063] Next, in step S240, the control device 3 stores the depth image captured by the depth sensor 2 in the storage unit 4 (creating and storing depth data).

[0064] Next, in step S250, the control device 3 calculates the prohibited overlaying points from the depth data using the detection system 100. Note that the prohibited overlaying points may also be calculated on the cloud side.

[0065] Next, in step S260, the control device 3 transmits data on no-loading locations to the robot 200 based on the acquired depth data. Note that if the robot 200 calculates the no-loading locations, it is not necessary to transmit the depth data and data on no-loading locations. Also, if the cloud side calculates the no-loading locations, the depth data and data on no-loading locations are transmitted from the cloud side to the robot 200.

[0066] Next, in step S270, the control device 3 detects prohibited overloading cargo from the depth data and corrects the depth data (detects and corrects prohibited overloading cargo from the depth data).

[0067] Next, in step S280, the control device 3 calculates the loading rate from the corrected depth data and stores it in the storage unit 4.

[0068] Next, in step S290, the control device 3 turns on the power saving mode of the overhead traveling vehicle. This shuts down the main controller of the overhead traveling vehicle and leaves only functions such as waiting for triggers running. This saves power and makes it possible to obtain necessary data such as loading rate.

[0069] Next, another example of the processing of the overhead traveling vehicle will be described with reference to Fig. 12. Fig. 12 is a flowchart showing another example of the processing of the overhead traveling vehicle. Here, as an example of the processing when the measurement results (depth data, loading rate) obtained by measuring the inside of the luggage compartment are sent to the cloud, a case where the measurement results are sent when the entrance to the luggage compartment is opened will be described.

[0070] First, in step S300, the control device 3 determines whether or not it has detected that the entrance to the luggage compartment has been opened. If it has detected that the door has been opened (step S300: YES), the process proceeds to step S310. If it has not detected that the door has been opened (step S300: NO), the process returns to before step S300.

[0071] Next, in step S310, the control device 3 cancels the power saving mode of the overhead traveling vehicle.

[0072] Next, in step S320, the control device 3 transmits data such as depth data and loading rate to the cloud.

[0073] Next, in step S330, the control device 3 turns on the power saving mode of the overhead traveling vehicle, which makes it possible to save power when transmitting necessary data such as loading rate to the cloud.

[0074] (Variation 3) In the above-described second modification, an overhead traveling vehicle (not shown) that travels along the ceiling in the fore-and-aft direction of the vehicle and a depth sensor 2 disposed on the overhead traveling vehicle have been described. Because the overhead traveling vehicle travels in the fore-and-aft direction of the vehicle in the center in the width direction of the vehicle, both ends of the luggage compartment in the width direction of the vehicle may be difficult to photograph for the depth sensor 2. This reduces the accuracy of detecting the container 5, increasing the possibility of erroneously determining whether or not there is cargo that is prohibited from being overloaded, which may reduce the accuracy of calculating the loading ratio.

[0075] Therefore, in Modification 3, multiple depth sensors 2 are arranged at predetermined intervals in the vehicle width direction (fixed horizontal axis type), the depth sensors 2 are configured to be movable in the vehicle width direction (movable horizontal axis type), or the depth sensors 2 are configured to be swivellable in the vehicle width direction (swivel horizontally). This prevents the depth sensors 2 from having difficulty capturing images at both ends of the luggage compartment in the vehicle width direction. Below, we will explain the fixed horizontal axis type depth sensors, the movable horizontal axis type depth sensors, and the swivel horizontally type depth sensors. Note that in Modification 3, the differences from the embodiment will be mainly explained, and the same components will be assigned the same numbers and their explanations will be omitted.

[0076] First, the horizontal shaft fixed depth sensor will be described with reference to Figures 13A and 13B. Figure 13A is a front view of the horizontal shaft fixed depth sensor as viewed from the vehicle longitudinal direction. Figure 13B is a side view of the horizontal shaft fixed depth sensor as viewed from the vehicle transverse direction. As shown in Figures 13A and 13B, the horizontal shaft fixed depth sensor has five monocular cameras 2a, a rail 6, an overhead traveling vehicle 7, and a horizontal shaft 8. The rail 6 is located on the ceiling of the luggage compartment and extends in the vehicle longitudinal direction. The overhead traveling vehicle 7 is capable of self-propelling in the vehicle longitudinal direction along the rail 6. The horizontal shaft 8 is attached to the overhead traveling vehicle 7 and extends in the vehicle transverse direction. The five monocular cameras 2a are located on the horizontal shaft 8 at the vehicle transverse center, both ends, and a position between the center and both ends. This allows, for example, cameras 2a placed at each of the two widthwise ends of the vehicle to photograph both widthwise ends of the cargo compartment from above, so that both widthwise ends of the cargo compartment are not difficult to photograph from the cameras 2a, making it possible to prevent a decrease in the accuracy of detecting containers 5 and also to suppress a decrease in the accuracy of calculating the loading ratio.

[0077] Next, the horizontal axis moving depth sensor will be described with reference to Figures 14A and 14B. Figure 14A is a front view of the horizontal axis moving depth sensor as viewed from the vehicle longitudinal direction. Figure 14B is a side view of the horizontal axis moving depth sensor as viewed from the vehicle transverse direction. As shown in Figures 14A and 14B, the horizontal axis moving depth sensor has one monocular camera 2a (or a stereo camera), rails 6, overhead traveling vehicles 7, racks 9, and guides 9a. The rails 6 are disposed on the ceiling of the luggage compartment and extend in the vehicle longitudinal direction. The overhead traveling vehicles 7 are self-propelled in the vehicle longitudinal direction along the rails 6 and have rack drive gears 7a. The racks 9 are movable in the vehicle transverse direction by the rack drive gears 7a. The guides 9a are fixed to the overhead traveling vehicles 7 and extend in the vehicle transverse direction. The camera 2a is fixed to the racks 9 and can be guided by the guides 9a. When the rack 9 is moved in the vehicle width direction by the rack drive gear 7a, the camera 2a fixed to the rack 9 is guided by the guide 9a and becomes movable in the vehicle width direction. As a result, for example, the cameras 2a moved to positions at both ends in the vehicle width direction can photograph both ends of the luggage compartment in the vehicle width direction from above, and therefore both ends of the luggage compartment in the vehicle width direction do not become locations that are difficult for the cameras 2a to photograph, making it possible to prevent a decrease in the accuracy of detecting the container 5 and also to suppress a decrease in the accuracy of calculating the loading ratio.

[0078] Next, the oscillating depth sensor will be described with reference to Figures 15A and 15B. Figure 15A is a front view of the oscillating depth sensor as viewed from the longitudinal direction of the vehicle. Figure 15B is a side view of the oscillating depth sensor as viewed from the transverse direction of the vehicle. As shown in Figures 15A and 15B, the oscillating depth sensor includes a monocular camera 2a (or a stereo camera), rails 6, an overhead traveling vehicle 7, a rotating shaft 9b, and a driven gear 9c. The rails 6 are disposed on the ceiling of the luggage compartment and extend in the longitudinal direction of the vehicle. The overhead traveling vehicle 7 is self-propelled in the longitudinal direction of the vehicle along the rails 6 and includes a drive gear 7b and a bearing 7c. The rotating shaft 9b is attached to the overhead traveling vehicle 7 via the bearing 7c and extends in the longitudinal direction of the vehicle. The rotating shaft 9b is rotatable about its axis via the bearing 7c. A belt is wound around the driven gear 9c and the drive gear 7b, and the driven gear 9c is fitted to and fixed to a rotating shaft 9b. The camera 2a is fixed to the rotating shaft 9b via an arm 2b. When the driven gear 9c is rotated by the drive gear 7b, the rotating shaft 9b and the arm 2b rotate integrally, thereby enabling the camera 2a to swing in the vehicle width direction. As a result, for example, the optical axis, which is the center of the angle of view of the camera 2a, can be directed toward both ends of the luggage compartment in the vehicle width direction to capture images. This prevents the camera 2a from having difficulty capturing images of both ends of the luggage compartment in the vehicle width direction, making it possible to prevent a decrease in the accuracy of detecting the container 5 and to suppress a decrease in the accuracy of calculating the loading ratio.

[0079] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these embodiments. In other words, the present disclosure can be carried out in various forms without departing from its gist or main features. [Industrial Applicability]

[0080] The present disclosure is suitably used in vehicles equipped with a detection system that is required to detect a wide variety of equipment without increasing costs. [Explanation of symbols]

[0081] 1 vehicle 2 Depth Sensor 2a Monocular camera 3. Control device 4 Storage section 5 Container 5a Top part 5b Top part 6 Rail 7 Overhead vehicle 7a Rack drive gear 7b Drive gear 7c Bearing part 8 Horizontal axis 9 racks 9a Guide 9b Rotation axis 9c Driven gear 30 Control Unit 31 Acquisition Department 32 First image generation unit 33 Second image generation unit 34 Calculation section 35 Judgment section 100 Detection System 200 robots

Claims

1. a depth sensor that is used for transporting or storing luggage, that is disposed above an open-topped equipment, and that captures a depth image that represents a distance from the above-top position to the equipment; a first image generating unit that extracts a depth value of a top surface portion of the equipment from the depth image and generates a binarized image based on the extracted depth value; a second image generating unit that generates a frame-shaped image based on a width and a depth of the top surface of the equipment in the depth image; a calculation unit that calculates a score value indicating a ratio of an area of ​​an overlapping region where the binary image and the frame-shaped image overlap while sliding the frame-shaped image within the binary image; a determination unit that determines the presence, orientation, and area of ​​the device based on a slide position of the frame-shaped image when the calculated score value is equal to or greater than a predetermined value; Equipped with Detection system.

2. When the determination unit determines the presence of the equipment, it further determines whether the equipment is prohibited from being loaded, and when all or part of the equipment is prohibited from being loaded, it determines that there is cargo in the space above the prohibited cargo and calculates a loading rate that indicates how effectively the interior of the cargo compartment is being utilized. The detection system of claim 1 .

3. Further comprising a ceiling moving part that runs in a straight line along a ceiling located above the equipment, The depth sensor is disposed on the ceiling moving part. The detection system of claim 1 .

4. The depth sensor is further movable in a direction perpendicular to the linear direction. The detection system of claim 3 .

5. The depth sensor is further rotatable in a direction perpendicular to the linear direction. The detection system of claim 3 .

Images