Cargo space detection system, car space detection program, car space detection method, and trained model for corner detection

Abstract

To provide a luggage compartment detection system capable of accurately detecting a range of a luggage compartment.SOLUTION: A luggage compartment detection system includes: an acquisition part which acquires an image obtained by imaging an entire luggage compartment with a loaded luggage visible in a predetermined imaging direction, and including the entire luggage compartment; a corner detection part which detects at least one of a plurality of corners of the luggage compartment from the image acquired by the acquisition part, using a learned model for corner detection machine-learned by using teacher image data labeled with a location of at least one corner of the plurality of corners of the luggage compartment when viewed in the imaging direction on the image acquired for machine learning; and a luggage compartment detection part which detects a range of the luggage compartment using the location information on the at least one corner detected by the corner detection part.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a luggage compartment detection system, a luggage compartment detection program, a luggage compartment detection method, and a trained model for corner detection. [Background technology]

[0002] For example, the loading and unloading system described in Patent Document 1 discloses a technology for estimating the height of a truck's loading platform from the ground by image analysis using a machine learning model. The machine learning model is generated by machine learning using image data of a large number of trucks and the loading platform heights from the ground as training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-116140 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, the inventors of the present application discovered a problem that when attempting to detect the entire range of the cargo space through image analysis using the above-mentioned machine learning model, the positions of the cargo space within the many images used as training data are shifted from one another, and the sizes, shapes, and loading order of the luggage loaded in the cargo space are diverse, etc., which act as disturbances, making it impossible to accurately detect the range of the cargo space when analyzing images using the machine learning model. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms. [Form 1] A cargo space detection system for detecting the extent of a cargo space, comprising: an acquisition unit that acquires an image including the entire cargo space by imaging the entire cargo space in a predetermined imaging direction with loaded luggage visible; a corner detection unit that detects at least one of the multiple corners of the cargo space from the image acquired by the acquisition unit using a trained model for corner detection trained by machine learning using teacher image data in which the position of at least one of the multiple corners of the cargo space when viewed in the imaging direction is labeled for the image acquired for machine learning; and a cargo space detection unit that detects the extent of the cargo space using position information of the at least one corner detected by the corner detection unit, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo space is labeled in addition to the positions of the corners.

[0006] (1) According to one embodiment of the present disclosure, a luggage compartment detection system is provided, which detects the extent of a luggage compartment, and includes: an acquisition unit that acquires an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined imaging direction while loaded luggage is visible, a corner detection unit that detects at least one of the multiple corners of the luggage compartment from the image acquired by the acquisition unit using a trained model for corner detection trained by machine learning using teacher image data in which a position of at least one of the multiple corners of the luggage compartment as viewed in the imaging direction is labeled for the image acquired for machine learning; and a luggage compartment detection unit that detects the extent of the luggage compartment using position information of the at least one corner detected by the corner detection unit. According to this embodiment of the luggage compartment detection system, corners are detected using a trained model for corner detection that has previously learned the positions of the luggage compartment corners, and the luggage compartment area is detected from the position information of the detected corners. Therefore, compared to a case where a trained model that has learned the luggage compartment itself is used, the influence of disturbances in an image including the entire luggage compartment can be reduced, output errors in the trained model are less likely to occur, and the luggage compartment area can be detected more appropriately. (2) In the above-described cargo space detection system, the outer shape of the cargo space is rectangular when viewed in the imaging direction, and the trained model for corner detection may be a trained model trained using the teacher image data in which the positions of two corners diagonally opposite each other among the multiple corners of the cargo space are labeled. According to this form of cargo space detection system, the trained model for corner detection is trained using teacher image data in which the positions of two diagonally opposite corners are labeled, so that the width and height of the cargo space can be calculated with high accuracy with a small amount of learning. (3) In the cargo compartment detection system of the above aspect, the trained model for corner detection may be a trained model trained using the teacher image data in which the positions of the corners and the positions of the lower region below the cargo compartment are labeled as features in addition to the positions of the corners. According to this aspect of the cargo compartment detection system, the trained model for corner detection labels the positions of the lower region of the cargo compartment as a feature in addition to the positions of the corners, so that the range of the cargo compartment can be detected with higher accuracy. (4) The luggage compartment detection system of the above aspect may further include a loading rate calculation unit that calculates a loading rate of the luggage in the luggage compartment, and the loading rate calculation unit may calculate the loading rate using the range of the luggage compartment detected by the luggage compartment detection unit and luggage information including information on the dimensions of the luggage in the luggage compartment. According to the luggage compartment detection system of this aspect, the loading rate can be accurately calculated using the accurately calculated range of the luggage compartment and the luggage information. (5) In the luggage compartment detection system of the above aspect, the luggage information may be detected from the image acquired by the acquisition unit using a trained model for luggage detection that is machine-learned to learn features of the luggage. According to the luggage compartment detection system of this aspect, luggage information for calculating a load ratio can be suitably detected by the trained model for luggage detection. (6) In the luggage compartment detection system of the above aspect, when the corner detection unit is unable to correctly detect at least one of the multiple corners, the acquisition unit may acquire the image again, or the trained model for corner detection may be re-trained. According to this luggage compartment detection system, even if the corner is unable to be correctly detected, the acquisition unit acquires the image again, or the trained model for corner detection is re-trained, so that the area of ​​the luggage compartment can be calculated after the corner is correctly detected again. The present disclosure may be realized in various forms other than the luggage compartment detection system, such as a luggage compartment detection method, a luggage compartment detection program, a non-transitory recording medium storing such a program, a trained model for corner detection, etc. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a hardware configuration of a luggage compartment detection system according to a first embodiment of the present disclosure. [Diagram 2] 2 is a block diagram showing a functional configuration of the luggage compartment detection system; [Diagram 3] 1 is a side view showing a schematic diagram of a vehicle equipped with a luggage compartment; [Figure 4] 5 is a flowchart showing a processing procedure of a luggage compartment detection method. [Diagram 5] 11A and 11B are diagrams for explaining image processing in corner detection; [Figure 6] 11A and 11B are diagrams for explaining image processing in luggage compartment detection. [Figure 7] 1A and 1B are diagrams for explaining image processing in baggage detection; [Figure 8] 10 is a flowchart showing a processing procedure of a luggage compartment detection method in a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] A. First embodiment: A luggage compartment detection system 10 and a luggage compartment detection method according to a first embodiment of the present disclosure will be described with reference to Figs. 1 to 7. A1. The cargo compartment detection system 10 includes: [Hardware configuration of luggage compartment detection system 10] FIG. 1 is a block diagram showing a hardware configuration of a luggage compartment detection system 10 according to a first embodiment of the present disclosure. The luggage compartment detection system 10 according to the present embodiment is configured to include a mobile terminal 12 having a photographing function, such as a smartphone or a tablet. The luggage compartment detection system 10 according to the first embodiment detects a luggage compartment of a vehicle to be described later and luggage in the luggage compartment, and calculates a loading rate, which is the ratio of the luggage to the luggage compartment. The configuration of the vehicle and the details of the loading rate calculation will be described later. In this embodiment, the luggage compartment detection system 10 can be used by starting the application in a state where a dedicated application is installed in advance in the mobile terminal 12.

[0009] 1, the mobile terminal 12 is provided with a control unit 14. The control unit 14 includes a CPU (Central Processing Unit: processor) 16, a ROM (Read Only Memory) 18, a RAM (Random Access Memory) 20, a storage 22 as a storage unit, a communication interface 24, and an input / output interface 26. Each component is connected to each other via a bus 28 so as to be able to communicate with each other.

[0010] The CPU 16 is a central processing unit that executes various programs and controls each part. That is, the CPU 16 reads out a program from the ROM 18 or the storage 22, and executes the program using the RAM 20 as a working area. The CPU 16 controls each of the above components and performs various calculation processes according to the program recorded in the ROM 18 or the storage 22. The storage 22 stores a luggage compartment detection program.

[0011] The storage 22 also stores a trained model for corner detection that detects corners of the luggage compartment from an image including the luggage compartment. This trained model for corner detection is generated by machine learning using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for an image acquired for machine learning that includes the entire luggage compartment. In the teacher image data of the first embodiment, the positions of two corners of the luggage compartment that are diagonally opposite each other are labeled. Labeling is a machine learning process that performs labeling on a large amount of raw data obtained from an image, and in this embodiment, it means associating a label indicating the position of the corner with the coordinates corresponding to the position of the corner.

[0012] Furthermore, the storage 22 stores a trained model for luggage detection that detects luggage from an image of a luggage compartment that contains luggage. This trained model for luggage detection is a model that machine-learns the feature amounts of luggage from an image including the entire luggage compartment. The feature amounts of luggage are, for example, brightness that differs depending on the material of the luggage, such as iron, plastic, and paper such as cardboard. The trained model for luggage detection can detect the outer shape indicating the dimensions of multiple luggage based on such feature amounts of luggage.

[0013] The ROM 18 stores various programs and various data. The RAM 20 temporarily stores programs or data as a working area. The storage 22 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including an operating system and various data. In this embodiment, the ROM 18 and the storage 22 store a program for performing a loading rate calculation process, various data including data related to the vehicle, and the like.

[0014] The communication interface 24 is an interface for the luggage compartment detection system 10 to communicate with a server and other devices (not shown), and uses standards such as Ethernet (registered trademark), LTE, FDDI, and Wi-Fi (registered trademark).

[0015] A display screen 30 as a display unit, a microphone 32, a speaker 34, and a camera 36 are connected to the input / output interface 26. The display screen 30 is provided on the mobile terminal 12, and displays various information to the user. The display screen 30 also displays image data captured by a camera 36, ​​which will be described later. In this embodiment, as an example, the display screen 30 is a touch panel type, and is configured so that input can be received by touching the content displayed on the display screen 30.

[0016] The microphone 32 and the speaker 34 are provided in the mobile terminal 12 and are used when the user makes a call, etc. The microphone 32 can also be used when the user issues voice instructions, and the speaker 34 can also be used when notifying the user by voice. The camera 36 is provided in the mobile terminal 12 and is configured so that image data captured by the camera 36 is displayed on the display screen 30.

[0017] [Functional configuration of luggage compartment detection system 10] The luggage compartment detection system 10 realizes various functions using the above hardware resources. The functional configuration realized by the luggage compartment detection system 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the luggage compartment detection system 10. Each functional configuration is realized by the CPU 16 reading out a program stored in the storage 22 and executing the program.

[0018] As shown in FIG. 2, the luggage compartment detection system 10 includes, as its functional configuration, an acquisition unit 41, a corner detection unit 42, a luggage compartment detection unit 43, a luggage detection unit 44, a loading rate calculation unit 45, and a learning unit 46. The acquisition unit 41 acquires an image including the entire luggage compartment to be detected by the camera 36. This image is obtained by capturing an image of the entire luggage compartment in a predetermined imaging direction in a state in which the loaded luggage is visible. The corner detection unit 42 detects corners of the luggage compartment from the image of the luggage compartment acquired by the acquisition unit 41 using a trained model for corner detection stored in the storage 22. The corner detection unit 42 also outputs position information of the detected corner. Details regarding the output of the position information of the corner will be described later.

[0019] The luggage compartment detection unit 43 detects the range of the luggage compartment from the position information of the corners detected by the corner detection unit 42. Specifically, the luggage compartment detection unit 43 has a function of numerically outputting the range of the luggage compartment from the position information of the corners, and further cutting out the range of the luggage compartment from the image as a partial image.

[0020] The luggage detection unit 44 detects luggage information in the luggage compartment from the image of the luggage compartment acquired by the acquisition unit 41 using the trained model for luggage detection stored in the storage 22. The luggage information includes information on the dimensions of the luggage in the luggage compartment. The loading rate calculation unit 45 calculates the loading rate of the luggage in the luggage compartment. Specifically, the loading rate is calculated using the range of the luggage compartment detected by the luggage compartment detection unit 43 and the luggage information in the luggage compartment. The learning unit 46 creates an AI model using a neural network based on the captured images of corners, luggage, etc. Note that the specific processing of each functional unit will be described in detail in the luggage compartment detection method described later.

[0021] [Vehicle 50 configuration] 3 is a side view showing a vehicle 50 having a luggage compartment 51. Next, a configuration of the vehicle 50 having the luggage compartment 51 to be detected by the luggage compartment detection system 10 will be described. The vehicle 50 is, for example, a truck that transports a large amount of luggage 52 loaded thereon.

[0022] As shown in Fig. 3, luggage compartment 51 is located behind driver's cab 58 of vehicle 50, and is formed on luggage platform 57. When luggage 52 is being transported, luggage compartment 51 is covered by side walls 53, 54 in the front-rear direction (left-right direction in Fig. 3) of vehicle 50, a side wall 55 on the right side in the traveling direction, a lower door 56 located on the left side in the traveling direction, and an upper door (not shown).

[0023] The upper door, for example, is an integrated door with a substantially L-shaped cross section formed by a left side wall and a part of the ceiling, and rotates upward to open the luggage compartment 51. The lower door 56 rotates downward from the bottom position of the luggage platform 57 to open the luggage compartment 51. When the lower door 56 and the upper door are open, it is possible to capture an image of the entire luggage compartment 51 including the positions of the four corners of the luggage compartment 51 from the left side of the vehicle 50. In other words, it is possible to obtain an image including the entire luggage compartment 51 in a state in which the loaded luggage 52 is visible. In this embodiment, the predetermined imaging direction coincides with the direction in which the luggage compartment 51 is viewed from the left side of the vehicle 50, as shown in FIG. 3. The outer shape of the luggage compartment 51 is rectangular when viewed in the imaging direction.

[0024] A2. Luggage compartment detection method by luggage compartment detection system 10: Next, a luggage compartment detection method executed by the luggage compartment detection system 10 will be described with reference to Figs. 4 to 7. Fig. 4 is a flowchart showing the processing procedure of the luggage compartment detection method. As shown in Fig. 4, the luggage compartment detection method includes an acquisition step (step 100, hereinafter "step" will be abbreviated as "S"), a corner detection step (S200), a luggage compartment detection step (S300), a baggage detection step (S400), and a loading rate calculation step (S500), and each of these steps is executed in order.

[0025] In the acquisition step (S100), a user of the system photographs the entire luggage compartment 51 with the camera 36, ​​and an image Ia (see FIG. 3, hereinafter also simply referred to as "full image Ia of the luggage compartment 51") including the entire luggage compartment 51 is acquired by the function of the acquisition unit 41. In the corner detection step (S200), the corner detection unit 42 detects corners of the luggage compartment 51 from the full image Ia of the luggage compartment 51 acquired in the acquisition step (S100) (see FIGS. 3 and 5) using a trained model for corner detection.

[0026] Fig. 5 is a diagram for explaining image processing in corner detection. As shown in Fig. 5, in this embodiment, of the four corners of the luggage compartment 51, two corners, an upper left corner Ic1 and a lower right corner Ic2, which are diagonally opposite each other in a left side view, are detected. In Fig. 5, the upper left corner Ic1 and the lower right corner Ic2 are illustrated surrounded by a two-dot chain line frame. In this embodiment, the "corner" means not only the vertices of the four corners of the rectangle representing the luggage compartment 51, but also a rectangle including parts of two line segments that intersect at approximately right angles with each other and have the vertices as end points.

[0027] In the luggage compartment detection process (S300), the luggage compartment detection unit 43 detects the range of the luggage compartment 51 from the position information of the corners Ic1 and Ic2 detected in the corner detection process (S200). In FIG. 5, when the horizontal direction is the X-axis and the vertical direction perpendicular to the X-axis is the Y-axis, the "position information of the corners Ic1 and Ic2" is the "X and Y coordinates of the corner points C1 and C2" shown in FIG. 5. Here, the X and Y coordinates of the upper left corner point C1 are (X1, Y1), and the X and Y coordinates of the lower right corner point C2 are (X2, Y2). In this embodiment, the corner points C1 and C2 are calculated by defining the center coordinates of each corner Ic1 and Ic2 of the rectangle as the corner points C1 and C2. The coordinates of each corner point C1 and C2 are calculated as the "position information of the corners Ic1 and Ic2".

[0028] The luggage compartment detection unit 43 extracts the outline of the luggage compartment 51 from the X, Y coordinates of two diagonal corner points C1, C2. Specifically, a rectangular image with the upper left corner point C1 (X1, Y1) and the lower right corner point C2 (X2, Y2) as diagonal corners is cut out as the outline of the luggage compartment 51. FIG. 6 is a diagram for explaining image processing in luggage compartment detection, and shows the cut-out luggage compartment image Ir. The luggage compartment image Ir is represented by a rectangular area surrounded by four vertices, the upper left corner point C1 (X1, Y1), the lower right corner point C2 (X2, Y2), the lower left corner point C3 (X1, Y2), and the upper right corner point C4 (X2, Y1).

[0029] In the luggage detection process (S400), the luggage detection unit 44 detects luggage information in the luggage compartment 51 from the overall image Ia of the luggage compartment 51 acquired in the acquisition process (S100) using a trained model for luggage detection. FIG. 7 is a diagram for explaining image processing in luggage detection, and shows an image Ib of a detected luggage 52. In FIG. 7, the image Ib of each luggage 52 is illustrated surrounded by a two-dot chain line frame. As shown in FIG. 7, in the luggage detection process (S400), the outer shape of each luggage 52 is detected as the image Ib of the multiple luggage 52.

[0030] In the loading rate calculation step (S500), the loading rate calculation unit 45 calculates the loading rate based on the luggage compartment image Ir detected in the luggage compartment detection step (S300) and the images Ib of the multiple luggage 52 detected in the luggage detection step (S400). As a specific process, the loading rate can be calculated by filling the images Ib of all the multiple luggage 52 loaded in the vehicle with black and dividing the number of filled pixels by the number of pixels of the luggage compartment image Ir.

[0031] In addition, the trained model for corner detection used in the above-mentioned cargo space detection system 10 that detects the range of the cargo space 51 can be used for other purposes in addition to calculating the loading rate, such as determining whether the vehicle 50 is parked in the correct position by comparing the external position of the detected cargo space 51 with the parking position of the vehicle 50.

[0032] A3.Effects: (1) According to the luggage compartment detection system 10 and luggage compartment detection method of the first embodiment, the corners Ic1 and Ic2 of the luggage compartment 51 can be detected using a trained model for corner detection that has previously learned the characteristics of the corners Ic1 and Ic2, and the range of the luggage compartment 51 is detected from the position information of the detected corners Ic1 and Ic2 (X, Y coordinates of the corner points C1 and C2). In the case of a trained model that has trained the luggage compartment 51 itself, various luggage 52 is stored in the luggage compartment 51 and the loading order is not determined, which causes disturbance when learning the external shape of the luggage compartment 51, and there is a problem that the accuracy of outputting the external shape of the luggage compartment 51 is reduced. In this regard, the corners Ic1 and Ic2 of the luggage compartment 51 are smaller than the entire luggage compartment 51 and are less susceptible to disturbances, so that the range of the luggage compartment 51 can be detected more accurately by using a trained model for corner detection that has learned the corners Ic1 and Ic2 to detect the luggage compartment 51.

[0033] (2) In addition, since the loading rate of the luggage 52 is calculated based on the accurately detected range of the luggage compartment 51, the loading rate can be calculated more accurately.

[0034] (3) The trained model for corner detection used in the cargo room detection system 10 and the cargo room detection method of the first embodiment described above is generated by machine learning using teacher image data in which the positions of two corners Ic1, Ic2 that are diagonally opposite each other of the cargo room 51 are labeled for an image including the entire cargo room 51. By labeling the positions of the two corners that are diagonally opposite each other, the width and height of the cargo room 51 can be calculated with high accuracy with a small amount of learning.

[0035] B. Second embodiment: Next, a luggage compartment detection system 10 and a luggage compartment detection method according to a second embodiment of the present disclosure will be described with reference to Fig. 8. Note that the same reference numerals are used for configurations substantially similar to those of the first embodiment, and descriptions thereof will be omitted. The system configuration and functional configuration of the luggage compartment detection system 10 of the second embodiment are substantially similar to those of the luggage compartment detection system 10 of the first embodiment.

[0036] The second embodiment differs from the first embodiment in that, in addition to the positions of two corners diagonally opposite to each other, among corners Ic1 and Ic2 of the luggage compartment 51, the entire luggage compartment 51 and the position of a lower door 56 of the luggage compartment 51 are labeled as teacher image data for the trained model for corner detection. The "lower door 56" corresponds to the "lower region" below the luggage compartment 51.

[0037] 8 is a flowchart showing the processing procedure of the luggage compartment detection method in the second embodiment. In the second embodiment, instead of the processing in the corner detection step (S200) in the first embodiment, S201, S202, and S203 are executed, and the other processing (S100, S300, S400, and S500) is the same, so the description will be omitted.

[0038] 8, in S201, the corner detection unit 42 detects corners Ic1 and Ic2 (see FIG. 5), the lower door 56, and the luggage compartment 51 from the entire image Ia of the luggage compartment 51 acquired in the acquisition step (S100) using a trained model for corner detection. Next, in S202, the control unit 14 determines whether the detection is correct or not.

[0039] Here, "when not detected correctly" refers to, for example, a case where detection of two corners Ic1, Ic2 is requested but only one is detected, or a case where there is a discrepancy larger than a preset threshold between the width (or length) of the luggage compartment 51 calculated from the detected corners Ic1, Ic2 and the width (or length) of the luggage compartment 51 calculated from the detected luggage compartment 51.

[0040] If it is determined in S202 that the detection has been correct (S202: YES), the process proceeds to S300. On the other hand, if it is determined in S202 that the detection has not been correct (S202: NO), the process proceeds to S203, where re-shooting or re-learning is performed. Specifically, when re-shooting, the CPU 16 guides the user to take another image of the luggage compartment 51 with the camera 36. When re-shooting, it is preferable that the shooting position, shooting angle, focus, etc. are appropriately corrected.

[0041] Furthermore, when re-learning is performed, the CPU 16 causes the learning unit 46 to acquire various captured images again and perform re-learning to update the existing trained model. After the process of S203, the process returns to the process of S100. Note that in S203, first, re-capture is performed, and if corners are still not correctly detected even after re-capturing a predetermined number of times, re-learning may be performed.

[0042] According to the luggage compartment detection system 10 and luggage compartment detection method of the second embodiment, it is possible to achieve the same effects as those of the first embodiment. Furthermore, the luggage compartment 51 is detected using teacher image data in which the positions of the two corners Ic1 and Ic2 of the luggage compartment 51 that are diagonally opposite each other, as well as the entire luggage compartment 51 and the position of the lower door 56 of the luggage compartment 51 are labeled. The lower door 56 is an element that has a uniform color and is less susceptible to disturbance. By using such additional elements, it is possible to more accurately calculate the range and loading rate of the luggage compartment 51.

[0043] C. Other embodiments: (C1) In the cargo room detection system 10 and cargo room detection method of each of the above embodiments, the trained model for corner detection uses teacher image data in which the positions of two corners Ic1 and Ic2 of the cargo room 51 that are diagonally opposite each other are labeled for the entire image Ia of the cargo room 51, but the position of the corners to be labeled may be one or may be three or more. When labeling the position of only one corner, the positions of the other corners may be detected by extracting from the corner features, for example, that at corners Ic1 and Ic2, the two sides that make up the cargo room 51 intersect at about 90°, and an area of ​​about 270° exists as a portion that does not make up the cargo room 51.

[0044] (C2) In addition, to detect the positions of other corners, as described in the second embodiment, the width and height of the luggage compartment 51 may be detected by appropriately combining with learning of the outer shape of the luggage compartment 51 and the lower door 56.

[0045] (C3) The luggage compartment detection system 10 and luggage compartment detection method of each of the above embodiments can be used by activating a dedicated application installed in advance on the mobile terminal 12. Alternatively, the user's smartphone or tablet may be configured to only take an image of the luggage compartment 51 and transmit the image to a general computer having a processor and memory as a system management terminal, and various programs may be executed in the destination computer to perform processes such as corner detection, luggage compartment detection, and loading rate calculation. In this case, the processing results may be returned to the user's smartphone or tablet.

[0046] (C4) In the luggage compartment detection system 10 and luggage compartment detection method of each of the above embodiments, the luggage compartment 51 of the vehicle 50 is the detection target, but the luggage compartment 51 is not limited to what the vehicle 50 has. The luggage compartment 51 may be, for example, a chute in which parts are appropriately stored and supplied at a production factory for the vehicle 50 parts. When the luggage compartment 51 is a chute, the part corresponding to the lower door 56 in the above second embodiment can be implemented as a space between the bottom of the chute and the floor on which the chute is placed. This space can be distinguished from the luggage compartment 51 part of the chute, and therefore can be learned as a feature separate from the luggage compartment 51 part.

[0047] (C5) In the above-described embodiments, the cargo room detection system 10 and the cargo room detection method have a loading rate calculation unit 45 and perform loading rate calculation. However, the cargo room detection system 10 and the cargo room detection method may not have the loading rate calculation unit 45 and may only detect the cargo room 51 without calculating the loading rate.

[0048] (C6) In each of the above embodiments of the luggage compartment detection system 10, the acquisition unit 41 acquires an image including the entire luggage compartment to be detected using the camera 36 possessed by the luggage compartment detection system 10, but an image captured by another camera may also be acquired via the communication interface 24.

[0049] The present disclosure is not limited to the above-mentioned embodiments, and can be realized in various configurations without departing from the spirit of the present disclosure. For example, the technical features in each embodiment corresponding to the technical features in each form described in the Summary of the Invention column can be appropriately replaced or combined in order to solve some or all of the above-mentioned problems or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0050] 10... luggage compartment detection system, 12... mobile terminal, 14... control unit, 16... CPU, 18... ROM, 20... RAM, 22... storage, 24... communication interface, 26... input / output interface, 28... bus, 30... display screen, 32... microphone, 34... speaker, 36... camera, 41... acquisition unit, 42... corner detection unit, 43... luggage compartment detection unit, 44... luggage detection unit, 45... loading rate calculation unit, 46... learning unit, 50... vehicle, 51... luggage compartment, 52... luggage, 53, 54, 55... side wall, 56... lower door (lower area), 57... luggage platform, C1... upper left corner point, C2... lower right corner point, C3... lower left corner point, C4... upper right corner point, Ia... image including entire luggage compartment, Ib... luggage image, Ic1... upper left corner, Ic2... lower right corner, Ir... luggage compartment image

Claims

1. A luggage compartment detection system for detecting a luggage compartment range, an acquisition unit that acquires an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined imaging direction while the loaded luggage is visible; a corner detection unit that detects at least one of the multiple corners of the luggage compartment from the image acquired by the acquisition unit using a trained model for corner detection that has been machine-learned using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for the image acquired for machine learning; and a luggage compartment detection unit that detects the range of the luggage compartment by using position information of the at least one corner detected by the corner detection unit; Equipped with A cargo compartment detection system, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo compartment is labeled in addition to the position of the corners.

2. The outer shape of the luggage compartment is rectangular when viewed in the imaging direction, The cargo compartment detection system of claim 1, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the positions of two corners diagonally opposite each other among the multiple corners of the cargo compartment are labeled.

3. A loading rate calculation unit is further provided for calculating a loading rate of the luggage in the luggage compartment, The loading rate calculation unit is 3. The luggage compartment detection system according to claim 1, further comprising: a luggage compartment detection unit that detects a range of the luggage compartment and a luggage information including information on a size of the luggage in the luggage compartment, and the luggage compartment detection unit detects the range of the luggage compartment and calculates the loading rate.

4. The luggage compartment detection system of claim 3 , wherein the luggage information is detected from the image acquired by the acquisition unit using a trained model for luggage detection that has been machine-learned to detect features of the luggage.

5. When the corner detection unit fails to correctly detect at least one of the plurality of corners, The luggage compartment detection system according to claim 1 or 2, wherein the image is acquired again by the acquisition unit, or re-learning of the trained model for corner detection is performed.

6. A luggage compartment detection program for detecting a luggage compartment range, a function of acquiring an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined capturing direction in a state in which the loaded luggage is visible; a function of detecting at least one of the multiple corners of the luggage compartment from the acquired image using a trained model for corner detection that has been machine-learned using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for the image acquired for machine learning; and A function of detecting the range of the luggage compartment using position information of the at least one corner detected; The above is realized by a computer. A cargo compartment detection program, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo compartment is labeled in addition to the position of the corner.

7. A method for detecting a range of a luggage compartment using a luggage compartment detection system, comprising: The luggage compartment detection system includes an acquisition unit, a corner detection unit, and a luggage compartment detection unit, an acquisition step of acquiring an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined imaging direction by the acquisition unit while the luggage is visible; a corner detection process in which the corner detection unit detects at least one of the multiple corners of the luggage compartment from the image acquired by the acquisition unit using a trained model for corner detection trained by machine learning using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for the image acquired for machine learning; a luggage compartment detection step of detecting a range of the luggage compartment by the luggage compartment detection unit using position information of the at least one corner detected by the corner detection unit; Equipped with A cargo compartment detection method, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo compartment is labeled in addition to the position of the corner.

Images