Cargo compartment detection system, cargo compartment detection program, cargo compartment detection method, and learning completed model for corner detection

Abstract

The present application provides a cargo compartment detection system capable of detecting the range of a cargo compartment with high precision. The cargo compartment detection system includes: an acquisition unit that acquires an image containing the entire cargo compartment, the image being obtained by photographing the entire cargo compartment in a state in which the loaded cargo can be visually recognized from a predetermined photographing direction; a corner detection unit that detects at least one corner of a plurality of corners of the cargo compartment from the image acquired by the acquisition unit using a learning completed model for corner detection, the learning completed model for corner detection being a learning completed model obtained by machine learning using training image data, the training image data being image data in which the position of at least one corner of a plurality of corners of the cargo compartment when viewed from the photographing direction is labeled for images acquired for machine learning; and a cargo compartment detection unit that detects the range of the cargo compartment using position information of the at least one corner detected by the corner detection unit.

Description

Technical Field

[0001] This disclosure relates to a cargo compartment inspection system, a cargo compartment inspection procedure, a cargo compartment inspection method, and a learned model for corner inspection. Background Technology

[0002] For example, in the loading and unloading system described in Patent Document 1, a technique is disclosed that uses a machine learning model to estimate the ground clearance of the truck's loading platform through image analysis. The machine learning model is generated by using image data of multiple trucks and the ground clearance of the loading platform as training data.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-116140 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] Here, the inventors of this application have discovered the following technical problem: when trying to detect the entire cargo compartment range by using image analysis with a machine learning model as described above, the displacement of cargo compartment positions within the images used as training data, as well as the variety of sizes, shapes, and loading orders of the goods loaded in the cargo compartments, can become interference, making it impossible to detect the cargo compartment range with high accuracy when using a machine learning model for image analysis.

[0008] Means for solving technical problems

[0009] This disclosure can be implemented in the following ways.

[0010] (1) According to one aspect of the present disclosure, a cargo compartment detection system is provided. The cargo compartment detection system comprises: an acquisition unit that acquires an image containing the entire cargo compartment, the image being obtained by capturing the entire cargo compartment in a state where the loaded goods can be visually identified along a predetermined shooting direction; a corner detection unit that uses a corner detection learning model to detect at least one corner from a plurality of corners of the cargo compartment from the image acquired by the acquisition unit, the corner detection learning model being a learned model obtained by machine learning using training image data, the training image data being image data for which the positions of at least one corner from the plurality of corners of the cargo compartment, when viewed from the shooting direction, are marked for the image acquired for machine learning purposes; and a cargo compartment detection unit that uses the position information of the at least one corner detected by the corner detection unit to detect the range of the cargo compartment.

[0011] The cargo compartment detection system based on this method can use a pre-learned corner detection model that has learned the positions of the cargo compartment corners to detect corners, and then detect the cargo compartment range based on the detected corner position information. Therefore, compared with the case of using a pre-learned model that learns the cargo compartment itself, it can reduce the influence of interference in the image containing the entire cargo compartment, the learned model is less prone to output errors, and it can more appropriately detect the cargo compartment range.

[0012] (2) In the above-described cargo compartment detection system, the cargo compartment may be rectangular in shape when viewed from the shooting direction, and the corner detection learning model may be a learning model obtained by learning from the training image data in which the positions of two corners that are diagonally opposite each other among the plurality of corners of the cargo compartment are marked.

[0013] According to this method, the corner detection model is a learned model obtained by learning from training image data in which the positions of two corners located at opposite corners are marked. Therefore, it can calculate the width and height of the cargo box with high accuracy with less learning.

[0014] (3) In the cargo compartment detection system described above, the corner detection learning model can also be a learned model obtained by learning from the training image data, which not only marks the positions of the corners but also marks the positions of the lower region below the cargo compartment. According to this cargo compartment detection system, the corner detection learning model marks not only the positions of the corners but also the positions of the lower region of the cargo compartment as feature values, thus enabling higher accuracy in detecting the range of the cargo compartment.

[0015] (4) In the cargo compartment detection system described above, a loading rate calculation unit may also be included, which calculates the loading rate of the goods in the cargo compartment. This loading rate calculation unit uses the range of the cargo compartment detected by the cargo compartment detection unit and goods information, including information related to the size of the goods in the cargo compartment, to calculate the loading rate. According to this cargo compartment detection system, the loading rate can be accurately calculated using the precisely calculated range of the cargo compartment and the goods information.

[0016] (5) In the cargo compartment detection system described above, the cargo information may also be detected from the image acquired by the acquisition unit using a cargo detection learned model obtained by machine learning of the cargo's features. According to this cargo compartment detection system, cargo information used for calculating the loading rate can be appropriately detected using the cargo detection learned model.

[0017] (6) In the cargo compartment detection system described above, if the corner detection unit fails to correctly detect at least one of the plurality of corners, the acquisition unit re-acquires the image, or performs relearning of the corner detection learning model. According to this cargo compartment detection system, even if a corner is not correctly detected, the acquisition unit re-acquires the image, or performs relearning of the corner detection learning model, thus enabling the calculation of the cargo compartment's range based on the correct detection of the corner again.

[0018] Furthermore, this disclosure can also be implemented in various ways other than a cargo compartment detection system. For example, it can be implemented using a cargo compartment detection method, a cargo compartment detection program, a non-transient recording medium storing the relevant program, a learned model for corner detection, etc. Attached Figure Description

[0019] Figure 1 This is a block diagram illustrating the hardware configuration of the cargo compartment detection system according to the first embodiment of this disclosure.

[0020] Figure 2 This is a block diagram illustrating the functional structure of the cargo compartment detection system.

[0021] Figure 3 It is a schematic side view of a vehicle with a cargo box.

[0022] Figure 4 This is a flowchart illustrating the processing steps of the cargo compartment inspection method.

[0023] Figure 5 This is a diagram used to illustrate image processing in corner detection.

[0024] Figure 6 This diagram illustrates the image processing used in cargo compartment inspection.

[0025] Figure 7 This is a diagram used to illustrate image processing in cargo inspection.

[0026] Figure 8 This is a flowchart illustrating the processing steps of the cargo compartment inspection method in the second embodiment. Detailed Implementation

[0027] A. First implementation method:

[0028] Reference Figures 1 to 7 This describes the cargo compartment detection system 10 and cargo compartment detection method of the first embodiment of this disclosure.

[0029] A1. Composition of the cargo compartment detection system 10:

[0030] [Hardware configuration of the cargo compartment detection system 10]

[0031] Figure 1 This is a block diagram showing the hardware configuration of the cargo compartment detection system 10 according to the first embodiment of this disclosure. Furthermore, the cargo compartment detection system 10 of this embodiment is configured to include a portable terminal 12 with a camera function, such as a smartphone or tablet computer. The cargo compartment detection system 10 of the first embodiment detects the cargo compartment of the vehicle described later and the goods inside the cargo compartment, and calculates the proportion of goods occupied in the cargo compartment, i.e., the loading rate. Details regarding the vehicle configuration and the loading rate calculation will be described later. In this embodiment, with a dedicated application pre-installed on the portable terminal 12, the cargo compartment detection system 10 can be utilized by launching the application.

[0032] like Figure 1 As shown, a control unit 14 is provided in the portable terminal 12. The control unit 14 is configured to include a CPU (Central Processing Unit) 16, a ROM (Read Only Memory) 18, a RAM (Random Access Memory) 20, a memory 22 as a storage unit, a communication interface 24, and an input / output interface 26. Each component is communicatively connected to each other via a bus 28.

[0033] CPU 16 is the central processing unit, executing various programs and controlling various components. Specifically, CPU 16 reads programs from ROM 18 or memory 22, using RAM 20 as its working area to execute the programs. CPU 16 performs control and various arithmetic operations according to the programs recorded in ROM 18 or memory 22. The cargo compartment detection program is stored in memory 22.

[0034] Additionally, memory 22 stores a learned corner detection model for detecting corners of the cargo compartment from an image containing the cargo compartment. This learned corner detection model is generated through machine learning using training image data. This training image data is image data in which the position of at least one of multiple corners of the cargo compartment, viewed from the shooting direction, is marked for an image acquired for machine learning purposes that includes the entire cargo compartment. In the training image data of the first embodiment, the positions of two corners of the cargo compartment that are diagonally opposite each other are marked. The term "marking" refers to the machine learning process of marking a large amount of raw data obtained from the image. In this embodiment, it means establishing a correlation between the identifier representing the position of the corner and the coordinates corresponding to the position of the corner.

[0035] Furthermore, memory 22 stores a learned cargo detection model for detecting cargo from an image containing a cargo compartment. This learned cargo detection model is a machine learning model that performs machine learning on the features of the cargo within an image containing the entire cargo compartment. The features of the cargo refer to, for example, the varying brightness depending on the material of the cargo, such as iron, plastic, or corrugated paper. Based on these features, the learned cargo detection model can detect the shape and dimensions of multiple cargo items.

[0036] ROM 18 stores various programs and data. RAM 20 serves as a working area for temporary storage of programs or data. Memory 22, consisting of an HDD (Hard Disk Drive) or SSD (Solid State Drive), stores various programs, including the operating system, and various data. In this embodiment, ROM 18 and memory 22 store programs for performing load rate calculations, as well as various data, including vehicle-related data.

[0037] Communication interface 24 is an interface for enabling the cargo compartment detection system 10 to communicate with servers and other devices not shown, such as using standards such as Ethernet (registered trademark), LTE, FDDI, and Wi-Fi (registered trademark).

[0038] The input / output interface 26 is connected to a display screen 30 (which serves as a display unit), a microphone 32, a speaker 34, and a camera 36. The display screen 30 is installed in the portable terminal 12 and displays various information to the user. Additionally, image data captured by the camera 36 (described later) is displayed on the display screen 30. Furthermore, in this embodiment, as an example, the display screen 30 is a touch panel type, configured to accept input by touching the content displayed on the display screen 30.

[0039] Microphone 32 and speaker 34 are respectively installed in the portable terminal 12 for use when the user makes calls, etc. In addition, microphone 32 can also be used when the user issues instructions by voice, and speaker 34 can also be used when notifying the user by voice. Camera 36 is installed in the portable terminal 12 and configured to display the image data captured by camera 36 on display screen 30.

[0040] [Functional Composition of Cargo Box Detection System 10]

[0041] The cargo compartment detection system 10 uses the aforementioned hardware resources to implement various functions. (Refer to...) Figure 2 Explain the functional composition of the cargo compartment detection system 10. Figure 2This is a block diagram showing the functional configuration of the cargo compartment detection system 10. Furthermore, each functional configuration is implemented by the CPU 16 reading the program stored in the memory 22 and executing that program.

[0042] like Figure 2 As shown, the cargo compartment detection system 10 is configured to include an acquisition unit 41, a corner detection unit 42, a cargo compartment detection unit 43, a cargo detection unit 44, a loading rate calculation unit 45, and a learning unit 46. The acquisition unit 41 acquires an image of the entire cargo compartment containing the target cargo using a camera 36. This image is obtained by capturing the entire cargo compartment in a state where the loaded cargo can be visually identified, along a predetermined shooting direction. The corner detection unit 42 uses a corner detection learning model stored in the memory 22 to detect the corners of the cargo compartment from the image acquired by the acquisition unit 41. Furthermore, the corner detection unit 42 outputs the position information of the detected corners. Details related to the output of the corner position information will be described later.

[0043] The cargo compartment detection unit 43 detects the range of the cargo compartment based on the position information of the corners detected by the corner detection unit 42. Specifically, the cargo compartment detection unit 43 has the function of outputting the range of the cargo compartment based on the position information of the corners, and then cropping the range of the cargo compartment as a portion of the image from the image.

[0044] The cargo detection unit 44 uses the learned cargo detection model stored in the memory 22 to detect cargo information inside the cargo compartment from the image of the cargo compartment acquired by the acquisition unit 41. The cargo information includes information related to the size of the cargo inside the cargo compartment. The loading rate calculation unit 45 calculates the loading rate of the cargo inside the cargo compartment. Specifically, the loading rate is calculated using the range of the cargo compartment detected by the cargo compartment detection unit 43 and the cargo information inside the cargo compartment. The learning unit 46 generates an AI model by effectively utilizing a neural network based on the captured images of corners, cargo, etc. Furthermore, the specific processing of each functional unit will be described in detail in the cargo compartment detection method described later.

[0045] [Composition of Vehicle 50]

[0046] Figure 3 This is a schematic side view of a vehicle 50 equipped with a cargo box 51. Next, the configuration of the vehicle 50, which is equipped with a cargo box 51 that is the object of detection by the aforementioned cargo box detection system 10, will be described. The vehicle 50 is, for example, a truck that loads and transports a large quantity of goods 52.

[0047] like Figure 3 As shown, the cargo box 51 is located behind the cab 58 of the vehicle 50 and is formed on the loading platform 57. The cargo box 51 is positioned relative to the front-rear direction of the vehicle 50 when transporting goods 52. Figure 3Side walls 53 and 54 (shown in the left and right directions), side wall 55 located on the right side relative to the direction of travel, lower section 56 located on the left side relative to the direction of travel, and upper section cover (not shown).

[0048] In the upper section, for example, a portion of the left side wall and ceiling forms an integral door with an approximately L-shaped cross-section, which rotates upward to open the cargo compartment 51. The lower section 56 rotates downward from the bottom of the loading platform 57 to open the cargo compartment 51. With the lower section 56 and the upper section open, the entire cargo compartment 51, including the four corners, can be photographed from the left side of the vehicle 50. In other words, an image including the entire cargo compartment 51 can be obtained while the loaded cargo 52 is visually identifiable. In this embodiment, as... Figure 3 As shown, the predetermined shooting direction is consistent with the direction from which the cargo box 51 is viewed from the left side of the vehicle 50. The cargo box 51 is rectangular in shape when viewed from the shooting direction.

[0049] A2. Cargo compartment detection method based on cargo compartment detection system 10:

[0050] Next, refer to Figures 4-7 This describes the cargo compartment inspection method performed by the aforementioned cargo compartment inspection system 10. Figure 4 This is a flowchart illustrating the processing steps of a cargo compartment inspection method. For example... Figure 4 As shown, the cargo compartment inspection method includes an acquisition process (step 100, hereinafter referred to as "step" as "S"), a corner inspection process (S200), a cargo compartment inspection process (S300), a cargo inspection process (S400), and a loading rate calculation process (S500), which are executed sequentially.

[0051] In the acquisition process (S100), the system user takes a picture of the entire cargo compartment 51 using camera 36, ​​thereby acquiring an image Ia (see reference) containing the entire cargo compartment 51 using the function of acquisition unit 41. Figure 3 Hereinafter referred to as "the overall image Ia of the cargo compartment 51"). In the corner detection process (S200), the corner detection unit 42 uses the corner detection model learned in the corner detection process to obtain the overall image Ia of the cargo compartment 51 obtained in the acquisition process (S100) (see reference). Figure 3 , Figure 5 The corner of cargo compartment 51 was inspected.

[0052] Figure 5 This is a diagram used to illustrate image processing in corner detection. For example... Figure 5 As shown, in this embodiment, two corners of the cargo compartment 51 that are diagonally opposite each other when viewed from the left are detected: the upper left corner Ic1 and the lower right corner Ic2. Figure 5In the illustration, the upper left corner Ic1 and the lower right corner Ic2 are enclosed by a double-dotted line frame. In this embodiment, the term "corner" refers to a rectangle that includes not only the vertices of the four corners of the rectangle representing the cargo compartment 51, but also a portion of two line segments that intersect each other at approximately right angles with each vertex as their endpoints.

[0053] In the cargo compartment inspection process (S300), the cargo compartment inspection unit 43 detects the range of the cargo compartment 51 based on the position information of corners Ic1 and Ic2 detected in the corner inspection process (S200). Figure 5 In this context, when the horizontal direction is defined as the X-axis and the perpendicular direction intersecting the X-axis is defined as the Y-axis, the so-called "position information of corners Ic1 and Ic2" refers to... Figure 5 The X and Y coordinates of corner points C1 and C2 are shown. Here, the X and Y coordinates of the upper left corner point C1 are set to (X1, Y1), and the X and Y coordinates of the lower right corner point C2 are set to (X2, Y2). In this embodiment, corner points C1 and C2 are calculated by defining the center coordinates of each corner Ic1 and Ic2 of the rectangle as corner points C1 and C2. The coordinates of each corner point C1 and C2 are calculated as "position information of corner Ic1 and Ic2".

[0054] The cargo compartment inspection unit 43 extracts the outline of the cargo compartment 51 based on the X and Y coordinates of the two diagonal corner points C1 and C2. Specifically, it cuts out an image of a rectangle with the upper left corner point C1 (X1, Y1) and the lower right corner point C2 (X2, Y2) as diagonals, and uses this image as the outline of the cargo compartment 51. Figure 6 This diagram illustrates the image processing in cargo compartment detection, showing the cropped cargo compartment image Ir. The cargo compartment image Ir is represented by a rectangular region enclosed by four vertices: the top-left corner C1(X1, Y1), the bottom-right corner C2(X2, Y2), the bottom-left corner C3(X1, Y2), and the top-right corner C4(X2, Y1).

[0055] In the cargo inspection process (S400), the cargo inspection unit 44 uses a learned cargo inspection model to detect cargo information inside the cargo compartment 51 from the overall image Ia of the cargo compartment 51 acquired in the acquisition process (S100). Figure 7 This is a diagram used to illustrate image processing in cargo inspection, showing image Ib of the inspected cargo 52. Figure 7 In the image, each item 52 is enclosed in a double-dotted-line frame for illustration. For example... Figure 7 As shown, in the cargo inspection process (S400), the external shape of each cargo 52 is detected as an image Ib of multiple cargoes 52.

[0056] In the loading rate calculation process (S500), the loading rate calculation unit 45 calculates the loading rate based on the cargo box image Ir detected in the cargo box inspection process (S300) and the images Ib of multiple goods 52 detected in the goods inspection process (S400). Specifically, this can be done by blackening the images Ib of all the goods 52 on the vehicle and dividing the number of blackened pixels by the number of pixels in the cargo box image Ir.

[0057] In addition, the corner detection learning model used in the cargo box detection system 10 within the detection range of the cargo box 51 can be used for purposes other than calculating the loading rate, such as determining whether the vehicle 50 is parked in the correct position by comparing the detected external position of the cargo box 51 with the parking position of the vehicle 50.

[0058] A3. Effect:

[0059] (1) According to the cargo compartment detection system 10 and cargo compartment detection method of the first embodiment described above, corner detection learning models that have pre-learned the features of corners Ic1 and Ic2 of cargo compartment 51 can be used to detect corners Ic1 and Ic2, and the range of cargo compartment 51 can be detected based on the position information of the detected corners Ic1 and Ic2 (X and Y coordinates of corner points C1 and C2). In the case of learning the learning model of cargo compartment 51 itself, since various goods 52 are contained in cargo compartment 51 and the loading order is uncertain, it will become an interference when learning the shape of cargo compartment 51, resulting in a problem of reduced accuracy of outputting the shape of cargo compartment 51. In this respect, since the corners Ic1 and Ic2 of cargo compartment 51 are smaller than the entire cargo compartment 51, they are less affected by interference. Therefore, by using the corner detection learning model that has learned corners Ic1 and Ic2 to detect cargo compartment 51, the range of cargo compartment 51 can be detected with higher accuracy.

[0060] (2) In addition, since the loading rate of cargo 52 is calculated based on the range of cargo compartment 51 detected with high precision, the loading rate can be calculated more accurately.

[0061] (3) The corner detection learning model used in the cargo compartment detection system 10 and cargo compartment detection method of the first embodiment described above is generated by machine learning using training image data. This training image data is image data in which the positions of two diagonally opposite corners Ic1 and Ic2 of the cargo compartment 51 are marked for an image containing the entire cargo compartment 51. By marking the positions of the two diagonally opposite corners, the width and height of the cargo compartment 51 can be calculated with high accuracy with less learning.

[0062] B. Second implementation method:

[0063] Next, refer to Figure 8 This section describes the cargo compartment detection system 10 and cargo compartment detection method according to the second embodiment of this disclosure. Furthermore, components substantially the same as those in the first embodiment are marked with the same reference numerals, and their descriptions are omitted. The system configuration and functional configuration of the cargo compartment detection system 10 of the second embodiment are substantially the same as those of the cargo compartment detection system 10 of the first embodiment.

[0064] In the second embodiment, the difference from the first embodiment is that, as the training image data for the learned model for corner detection, in addition to marking the positions of the two diagonally opposite corners Ic1 and Ic2 of the cargo compartment 51, the entire cargo compartment 51 and the position of the lower section 56 of the cargo compartment 51 are also marked. The "lower section 56" is equivalent to the "lower region" below the cargo compartment 51.

[0065] Figure 8 This is a flowchart illustrating the processing steps of the cargo compartment inspection method in the second embodiment. In the second embodiment, S201, S202, and S203 are performed instead of the corner inspection process (S200) in the first embodiment. The other processes (S100, S300, S400, and S500) are the same, so their description is omitted.

[0066] like Figure 8 As shown, in S201, the corner detection unit 42 uses a learned corner detection model to detect corners Ic1 and Ic2 from the overall image Ia of the cargo box 51 acquired in the acquisition process (S100). Figure 5 ), lower section 56 and cargo compartment 51. Next, in S202, the control unit 14 determines whether the detection was correct.

[0067] Here, the so-called "situation of not being detected correctly" is imagined to be, for example, that although it is required to detect two corners Ic1 and Ic2, only one is detected, or that there is a deviation greater than a preset threshold between the width (or length) of the cargo compartment 51 calculated based on the detected corners Ic1 and Ic2 and the width (or length) of the cargo compartment 51 calculated based on the detected cargo compartment 51.

[0068] In S202, if the detection is determined to be correct (S202: Yes), the process proceeds to S300. Conversely, if the detection is determined to be incorrect in S202 (S202: No), the process proceeds to S203 to perform a re-shoot or relearning. Specifically, in the case of a re-shoot, the CPU 16 guides the user to take another picture of the cargo compartment 51 using the camera 36. During this re-shoot, it is preferable to appropriately adjust the shooting position, shooting angle, focus, etc.

[0069] Furthermore, in the case of relearning, the CPU 16, using the learning unit 46, re-acquires various captured images and relearns, updating the existing learned model. After processing in S203, it returns to processing in S100. Alternatively, in S203, re-capture can be performed first, and if the corner is still not correctly detected even after a specified number of re-captures, relearning can be performed.

[0070] The cargo compartment detection system 10 and cargo compartment detection method according to the second embodiment can achieve the same effect as the first embodiment described above. Furthermore, the cargo compartment 51 is detected using training image data that marks not only the positions of two diagonally opposite corners Ic1 and Ic2 of the cargo compartment 51, but also the entire cargo compartment 51 and the position of its lower section 56. The lower section 56 is an element of the same color that is less susceptible to interference. By additionally using such an element, the range and loading rate of the cargo compartment 51 can be calculated with high accuracy.

[0071] C. Other implementation methods:

[0072] (C1) In the cargo compartment detection system 10 and cargo compartment detection method of the above embodiments, the corner detection model uses training image data obtained by marking the positions of two corners Ic1 and Ic2 that are diagonally opposite each other in the image Ia of the entire cargo compartment 51. However, the marked corner positions can be one or more than three. In the case of marking only one corner position, the detection of other corner positions can be performed based on the following corner features, for example: in corners Ic1 and Ic2, the two sides constituting the cargo compartment 51 intersect at approximately 90°, and the area of ​​approximately 270° exists as a part that does not constitute the cargo compartment 51.

[0073] (C2) In addition, in the detection of other corner positions, the width and height of the cargo box 51 can be detected by appropriately combining the shape of the cargo box 51 and the learning of the lower part 56, as described in the second embodiment.

[0074] (C3) In the cargo compartment detection system 10 and cargo compartment detection method of the above embodiments, if a dedicated application is pre-installed on the portable terminal 12, it can be utilized by launching the application. Alternatively, it can be configured so that only images of the cargo compartment 51 are captured on the user's smartphone or tablet, and the images are sent to a general computer with a processor and memory, which serves as the system management terminal. Various programs are executed on the computer receiving the images, performing corner detection, cargo compartment detection, and loading rate calculation, among other processes. In this case, the processing results can also be returned to the user's smartphone or tablet.

[0075] (C4) The cargo compartment detection system 10 and cargo compartment detection method of the above embodiments take the cargo compartment 51 of the vehicle 50 as the detection object, but the cargo compartment 51 is not limited to the cargo compartment of the vehicle 50. For example, the cargo compartment 51 can also be a chute that appropriately stores and supplies parts in a manufacturing plant for vehicle 50 parts. In addition, when the cargo compartment 51 is a chute, in the second embodiment described above, the part corresponding to the lower part 56 can be implemented as the space between the bottom of the chute and the floor on which the chute is disposed. Since this space can be distinguished from the cargo compartment 51 part of the chute, it can be learned as a feature different from the cargo compartment 51 part.

[0076] (C5) In the cargo compartment detection system 10 and cargo compartment detection method of the above embodiments, it is provided that there is a loading rate calculation unit 45 and the loading rate calculation is performed. However, it may also be provided that there is no loading rate calculation unit 45, and the cargo compartment detection system 10 and cargo compartment detection method do not perform loading rate calculation but only detect the cargo compartment 51.

[0077] (C6) In the cargo compartment detection system 10 of the above embodiments, the acquisition unit 41 is configured to acquire an image of the entire cargo compartment containing the detection object through the camera 36 provided by the cargo compartment detection system 10, but it can also acquire an image taken by another camera through the communication interface 24.

[0078] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, technical features in each embodiment corresponding to the technical features in the various methods described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described technical problems, or to achieve some or all of the above-described effects. In addition, technical features can be appropriately deleted as long as they are not described as essential parts in this specification.

[0079] Explanation of reference numerals in the attached figures

[0080] 10: Cargo compartment detection system, 12: Portable terminal, 14: Control unit, 16: CPU, 18: ROM, 20: RAM, 22: Memory, 24: Communication interface, 26: Input / output interface, 28: Bus, 30: Display screen, 32: Microphone, 34: Speaker, 36: Camera, 41: Acquisition unit, 42: Corner detection unit, 43: Cargo compartment detection unit, 44: Cargo detection unit, 45: Loading rate calculation unit, 46: Learning unit, 50: Vehicle, 51: Cargo compartment, 52: Cargo, 53, 54, 55: Side wall, 56: Lower part (lower area), 57: Loading platform, C1: Top left corner, C2: Bottom right corner, C3: Bottom left corner, C4: Top right corner, Ia: Image containing the entire cargo compartment, Ib: Image of the cargo, Ic1: Top left corner, Ic2: Bottom right corner, Ir: Image of the cargo compartment.

Claims

1. A cargo compartment detection system for detecting the range of a cargo compartment, comprising: The acquisition unit acquires an image containing the entire cargo compartment, which is obtained by taking a picture of the entire cargo compartment in a state in which the loaded goods can be visually identified along a predetermined shooting direction; The corner detection unit uses a corner detection learned model to detect at least one corner of a plurality of corners of the cargo compartment from the image acquired by the acquisition unit. The corner detection learned model is a learned model obtained by machine learning using training image data. The training image data is image data for the image acquired for machine learning, in which the positions of at least one corner of the plurality of corners of the cargo compartment when viewed from the shooting direction are marked, and the positions of the lower region below the cargo compartment are marked. as well as The cargo compartment detection unit uses the position information of at least one corner detected by the corner detection unit to detect the range of the cargo compartment.

2. The cargo compartment detection system according to claim 1, wherein, The cargo box appears rectangular when viewed from the shooting direction. The corner detection learning model is a learning model obtained by learning from the training image data in which the positions of two corners that are diagonally opposite each other among the plurality of corners of the cargo compartment are marked.

3. The cargo compartment detection system according to claim 1 or 2, wherein, It also includes a loading rate calculation unit, which calculates the loading rate of the goods inside the cargo compartment. The loading rate calculation unit uses the range of the cargo compartment detected by the cargo compartment detection unit and cargo information, including information related to the size of the cargo in the cargo compartment, to calculate the loading rate.

4. The cargo compartment detection system according to claim 3, wherein, The cargo information is detected from the image acquired by the acquisition unit using a cargo detection model that has been trained by machine learning on the features of the cargo.

5. The cargo compartment detection system according to claim 1 or 2, wherein, If the corner detection unit fails to correctly detect at least one of the plurality of corners, the image is acquired again by the acquisition unit, or the learning model for corner detection is relearned.

6. A cargo compartment detection program for detecting the range of a cargo compartment, which enables a computer to perform the following functions: The function of acquiring an image containing the entire cargo compartment, which is obtained by taking a picture of the entire cargo compartment in a state in which the loaded goods can be visually identified along a predetermined shooting direction; The corner detection learning-complete model is used to detect at least one corner of a plurality of corners of the cargo compartment from the acquired image. The corner detection learning-complete model is a learning-complete model obtained by machine learning using training image data. The training image data is image data for the image acquired for machine learning, in which the positions of at least one corner of the plurality of corners of the cargo compartment when viewed from the shooting direction are marked, and the positions of the lower region below the cargo compartment are marked. as well as The function of detecting the range of the cargo compartment using the detected position information of at least one corner.

7. A method for detecting the range of a cargo compartment using a cargo compartment detection system. The cargo compartment inspection system includes an acquisition unit, a corner inspection unit, and a cargo compartment inspection unit. The cargo compartment inspection method includes the following steps: The acquisition process involves acquiring an image of the entire cargo compartment by means of the acquisition unit. This image is obtained by taking a picture of the entire cargo compartment in a state in which the loaded goods can be visually identified, along a predetermined shooting direction. The corner detection process involves using a corner detection learning model from the image acquired by the acquisition unit to detect at least one corner of the cargo compartment from a plurality of corners. The corner detection learning model is a learned model obtained by machine learning using training image data. The training image data is image data for the image acquired for machine learning, in which the positions of at least one corner of the cargo compartment from the shooting direction are marked, and the positions of the lower area below the cargo compartment are also marked. as well as The cargo compartment inspection process involves using the cargo compartment inspection unit to detect the range of the cargo compartment by using the position information of at least one corner detected by the corner inspection unit.

Images