Information processing device, information processing system, information processing method, and program

Abstract

The present invention provides an information processing device capable of suppressing cargo collapse of loads loaded onto a mobile vehicle. [Solution] The information processing device 10 includes a control unit 15 that notifies the user of notification information that helps suppress cargo collapse when a mobile body loaded with cargo moves, based on at least one of the movement conditions of the mobile body and the environmental conditions around the mobile body, as well as the state of the cargo loading.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing system, an information processing method, and a program.

Background Art

[0002] Conventionally, when a moving body moves, a technique for monitoring the collapse of a load loaded on the moving body is known. For example, Patent Document 1 discloses a load monitoring device for monitoring a load during transportation, particularly for monitoring the collapse of a load during transportation by a vehicle, ship, railway, etc., and reducing damage to the load.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, only the monitoring of load collapse is considered, and there is room for improvement in assisting the user before load collapse occurs so that load collapse is suppressed.

[0005] An object of the present disclosure is to provide an information processing apparatus, an information processing system, an information processing method, and a program capable of suppressing the collapse of a load loaded on a moving body.

Means for Solving the Problems

[0006] An information processing apparatus according to a first aspect for solving the above problems is a control unit that notifies a user of notification information for assisting in suppressing the collapse of a load accompanying the movement of the moving body based on at least one of the movement conditions of the moving body and the environmental conditions around the moving body and the loading state of the load when the moving body loaded with the load moves. It is equipped with.

[0007] Information processing systems from the second perspective are: The above information processing device and, An imaging device that captures the loading state and transmits the image obtained to the information processing device, It is equipped with.

[0008] The third perspective on information processing methods is: When a mobile body loaded with cargo is in motion, notification information is provided to the user to help suppress cargo collapse during the movement of the mobile body, based on at least one of the conditions under which the mobile body is moving, the environmental conditions surrounding the mobile body, and the state of the cargo loading. Includes.

[0009] The program from the fourth perspective is: In an information processing device, When a mobile body loaded with cargo is in motion, notification information is provided to the user to help suppress cargo collapse during the movement of the mobile body, based on at least one of the conditions under which the mobile body is moving, the environmental conditions surrounding the mobile body, and the state of the cargo loading. Perform an action that includes this. [Effects of the Invention]

[0010] According to an information processing device, information processing system, information processing method, and program according to one embodiment of the present disclosure, it is possible to suppress the collapse of cargo loaded on a moving body. [Brief explanation of the drawing]

[0011] [Figure 1] This block diagram shows a schematic configuration of an information processing system including an information processing device according to one embodiment of the present disclosure. [Figure 2] This is a flowchart illustrating a first example of an information processing method performed by the information processing device shown in Figure 1. [Figure 3]A flowchart for explaining a second example of an information processing method executed by the information processing apparatus of FIG. 1. [Figure 4] A flowchart for explaining a third example of an information processing method executed by the information processing apparatus of FIG. 1. [Figure 5] A flowchart for explaining a fourth example of an information processing method executed by the information processing apparatus of FIG. 1. [Figure 6] A flowchart for explaining a fifth example of an information processing method executed by the information processing apparatus of FIG. 1. [Figure 7] FIG. 1 is a first diagram for explaining an example of the processing by the information processing apparatus of FIG. 1. [Figure 8] FIG. 2 is a second diagram for explaining an example of the processing by the information processing apparatus of FIG. 1. [Figure 9] FIG. 3 is a third diagram for explaining an example of the processing by the information processing apparatus of FIG. 1. [Figure 10] FIG. 4 is a fourth diagram for explaining an example of the processing by the information processing apparatus of FIG. 1. [[ID=二十一]]

Embodiments for Carrying Out the Invention

[0012] Hereinafter, an embodiment of the present disclosure will be mainly described with reference to the accompanying drawings. The following description also applies to an information processing method and a program executed by the information processing apparatus 10 to which the present disclosure is applied.

[0013] FIG. 1 is a block diagram showing a schematic configuration of an information processing system 1 including an information processing apparatus 10 according to an embodiment of the present disclosure. With reference to FIG. 1, the outline of the information processing system 1 including the information processing apparatus 10 according to an embodiment of the present disclosure will be mainly described. The information processing system 1 has an imaging apparatus 20 in addition to the information processing apparatus 10.

[0014] [[ID=三十六]] In FIG. 1, for the sake of simplicity of explanation, only one information processing apparatus 10 and one imaging apparatus 20 are illustrated for each of the information processing apparatus 10 and the imaging apparatus 20, but the number of information processing apparatuses 10 and imaging apparatuses 20 included in the information processing system 1 may be two or more respectively. The information processing apparatus 10 and the imaging apparatus 20 are installed on a moving body described later and are communicably connected to each other based on an arbitrary communication standard in the moving body.

[0015] An outline of an embodiment will be described. The information processing apparatus 10 notifies a user of notification information for assisting in suppressing the collapse of a load accompanying the movement of a moving body based on at least one of the movement conditions of the moving body and the environmental conditions around the moving body and the loading state of the load when the moving body carrying the load moves.

[0016] In the present disclosure, the "moving body" includes, for example, a ship such as an offshore ship. The "load" includes, for example, steel cargo transported by a ship. The steel cargo includes, for example, steel coils. The "movement conditions" include conditions such as accelerations, speeds, rollings, pitchings, GM (Gravity Metacenter), trims, the number of loads loaded, and the loading weight of the load in six directions including the vertical direction and the traveling direction. The "environmental conditions" include meteorological conditions including states and phenomena of the atmosphere such as weather, temperature, and wind strength, and sea state conditions including states and phenomena of the ocean such as wave height.

[0017] In the present disclosure, the "loading state" includes, for example, the arrangement of the load on the moving body. The loading state may include the arrangement of the load when a shift occurs based on rotation and translational movement of the load, or may include the arrangement of the load that remains in the initial state loaded on the moving body without a shift occurring. The loading state may include the arrangement of the load when the shift progresses and a load collapse occurs. The "shift" means, for example, that the arrangement of the load loaded on the moving body changes from the initial arrangement as the load rotates, translates, or tilts as the moving body moves.

[0018] In this disclosure, “Notification Information” includes alert information and predictive information. “Alert Information” may, for example, simply indicate that cargo displacement has been detected or that there is a high probability that cargo displacement is occurring. “Predictive Information” may, for example, include first predictive information, second predictive information, and third predictive information. First predictive information indicates whether cargo displacement and cargo collapse are expected to occur during the future movement of the moving object. Second predictive information indicates a travel route in which cargo collapse is expected to be suppressed during the movement of the moving object. Third predictive information indicates at least one of the travel conditions and environmental conditions in which cargo collapse is expected to be suppressed during the movement of the moving object.

[0019] In this disclosure, “User” includes, for example, any operator who moves the mobile object while controlling it. The User also includes a crew member who navigates a vessel along a predetermined route while adjusting the vessel’s movement conditions.

[0020] The information processing device 10 is any electronic device installed on a mobile device that receives necessary input operations from the user and outputs necessary information to the user. The information processing device 10 may be any general-purpose electronic device such as a PC (Personal Computer) or a smartphone, or it may be another electronic device dedicated to the information processing system 1. The information processing device 10 is not limited to these, and may be one or multiple server devices that can communicate with each other.

[0021] The imaging device 20 is any electronic device that images the interior space of a warehouse or container where cargo is loaded on a moving object. The imaging device 20 includes a camera. The imaging device 20 images the cargo loaded on the moving object and generates an image. For example, the imaging device 20 images steel coils loaded in the cargo hold of a ship and generates an image. The imaging device 20 transmits the generated image to the information processing device 10.

[0022] Referring to Figure 1, an example of the configuration of the information processing device 10 included in the information processing system 1 will be mainly described. The information processing device 10 has a communication unit 11, a storage unit 12, an input unit 13, an output unit 14, and a control unit 15.

[0023] The communication unit 11 includes one or more communication interfaces that are communicatively connected to the imaging device 20. These communication interfaces correspond to any communication standard defined between the information processing device 10 and the imaging device 20. In one embodiment, the information processing device 10 is communicatively connected to the imaging device 20 via the communication unit 11. The communication unit 11 receives images generated by the imaging device 20 from the imaging device 20.

[0024] The storage unit 12 includes storage modules such as an HDD (Hard Disk Drive), SSD (Solid State Drive), EEPROM (Electrically Erasable Programmable Read-Only Memory), ROM (Read-Only Memory), and RAM (Random Access Memory). The storage unit 12 stores information necessary to realize the operation of the information processing device 10. The storage unit 12 also stores information obtained through the operation of the information processing device 10. For example, the storage unit 12 stores system programs, application programs, and various data obtained by any means such as communication.

[0025] The storage unit 12 may function as a main memory module, an auxiliary memory module, or a cache memory. The storage unit 12 is not limited to one built into the information processing device 10, and may also include an external storage module connected by a digital input / output port such as USB (Universal Serial Bus).

[0026] The input unit 13 includes one or more input interfaces that detect user input and acquire input information based on user operations. These input interfaces include physical keys, capacitive keys, a touchscreen integrated with the display of the output unit 14, an imaging module such as a camera, and a microphone that accepts voice input.

[0027] The output unit 14 includes one or more output interfaces that output information to notify the user. These output interfaces include a display that outputs information as an image, a speaker that outputs information as sound, and a vibrator that outputs information as vibration.

[0028] The control unit 15 includes one or more processors. In this disclosure, “processor” is a general-purpose processor or a dedicated processor specialized for a particular process, but is not limited to these. The control unit 15 is communicatively connected to each component constituting the information processing device 10 and controls the operation of the entire information processing device 10.

[0029] This section mainly describes an example of the configuration of an imaging device 20 included in the information processing system 1. The imaging device 20 includes a communication unit 21, a storage unit 22, an imaging unit 23, and a control unit 24.

[0030] The communication unit 21 includes one or more communication interfaces that are communicatively connected to the information processing device 10. These communication interfaces correspond to any communication standard defined between the information processing device 10 and the imaging device 20. In one embodiment, the imaging device 20 is communicatively connected to the information processing device 10 via the communication unit 21. The communication unit 21 transmits images generated by the imaging device 20 to the information processing device 10.

[0031] The storage unit 22 includes storage modules such as HDD, SSD, EEPROM, ROM, and RAM. The storage unit 22 stores information necessary to realize the operation of the imaging device 20. The storage unit 22 stores information obtained through the operation of the imaging device 20. For example, the storage unit 22 stores system programs, application programs, and various data acquired by any means such as communication.

[0032] The storage unit 22 may function as a main memory module, an auxiliary memory module, or a cache memory. The storage unit 22 is not limited to one built into the imaging device 20, and may also include an external storage module connected by a digital input / output port such as USB.

[0033] The imaging unit 23 includes an image sensor. The image sensor is composed of, for example, tens of thousands to millions or more light-detecting elements integrated onto a small chip. The image sensor includes CCDs (Charge Coupled Devices) and CMOSs ​​(Complementary Metal-Oxide Semiconductors).

[0034] The control unit 24 includes one or more processors. The control unit 24 is communicatively connected to each component of the imaging device 20 and controls the operation of the entire imaging device 20.

[0035] Figure 2 is a flowchart illustrating a first example of an information processing method performed by the information processing device 10 in Figure 1. The first example of an information processing method performed by the information processing device 10 in Figure 1 will be mainly explained with reference to Figure 2.

[0036] In step S101, the control unit 15 of the information processing device 10 periodically or irregularly acquires images of the cargo captured by the imaging device 20 from the imaging device 20 via the communication unit 11. The control unit 15 acquires multiple images of the cargo, which are repeatedly captured using the imaging device 20 while the moving object is in motion, in a time series.

[0037] In step S102, the control unit 15 of the information processing device 10 analyzes the multiple images acquired in step S101. By analyzing the images, the control unit 15 acquires information about the loading status of the cargo on the moving object. For example, the control unit 15 acquires information about the loading status of steel coils by analyzing images taken inside the cargo hold of a ship.

[0038] The control unit 15 performs image processing on each image acquired in step S101 using a Sobel filter that extracts contours from the image using the first derivative. The control unit 15 acquires contour images for each image acquired in step S101. The control unit 15 superimposes the acquired contour images.

[0039] At this time, if the loading state of the cargo changes and shifting occurs, the contours of one image and the other image will not overlap. For example, if a steel coil is rotating or translating, the contour of the steel coil will not overlap between one image and the other image. The control unit 15 identifies the cargo that has shifted by determining whether the contours of one image and the other image match or not. For example, the control unit 15 identifies a steel coil that has shifted based on rotation or translational movement.

[0040] In step S103, the control unit 15 of the information processing device 10 determines whether or not cargo displacement has occurred based on the loading state of the cargo obtained from the image analysis in step S102. If the control unit 15 determines that cargo displacement has occurred, it executes the process in step S104. If the control unit 15 determines that cargo displacement has not occurred, it terminates the process.

[0041] In step S104, the control unit 15 of the information processing device 10 determines whether the history of movement conditions immediately preceding the movement of the moving object satisfies the first condition. In this disclosure, the "first condition" includes, for example, movement conditions that are likely to cause cargo displacement. The first condition may be determined objectively based on past measured data relating the movement conditions of the moving object and the loading state of the cargo, or it may be determined subjectively based on the user's rules of thumb. If the control unit 15 determines that the history of movement conditions satisfies the first condition, it executes the process in step S105. If the control unit 15 determines that the history of movement conditions does not satisfy the first condition, it terminates the process.

[0042] In step S105, the control unit 15 of the information processing device 10 determines whether the environmental conditions on the route to which the moving object is scheduled to travel satisfy the second condition. In this disclosure, the "second condition" includes, for example, environmental conditions that are likely to cause cargo shifting. The second condition may be determined objectively based on past measured data relating the environmental conditions around the moving object and the loading state of the cargo, or it may be determined subjectively based on the user's rules of thumb. If the control unit 15 determines that the environmental conditions satisfy the second condition, it executes the process in step S106. If the control unit 15 determines that the environmental conditions do not satisfy the second condition, it terminates the process.

[0043] In step S106, if all conditions are met in the determination process of steps S103 to S105, the control unit 15 of the information processing device 10 notifies the user of alert information that helps suppress cargo shifting due to the movement of the moving object. For example, the control unit 15 notifies the user of alert information indicating that cargo shifting has been detected, in order to appeal to the user's sight, hearing, or touch, using the output unit 14. The control unit 15 issues an alert to the user regarding the detection of cargo shifting using the output unit 14.

[0044] Figure 3 is a flowchart illustrating a second example of the information processing method performed by the information processing device 10 in Figure 1. The second example of the information processing method performed by the information processing device 10 in Figure 1 will be mainly explained with reference to Figure 3.

[0045] In step S201, the control unit 15 of the information processing device 10 acquires the movement conditions of a mobile body loaded with cargo when it has previously traveled a predetermined route. For example, the control unit 15 acquires the movement conditions of a mobile body stored in the storage unit 12 as a record of the mobile body's past movements by referring to the storage unit 12. This recording of movement conditions may be based on past input operations by the user using the input unit 13 of the information processing device 10, or it may be based on past reception operations from various sensor devices of the mobile body using the communication unit 11 of the information processing device 10.

[0046] In step S202, the control unit 15 of the information processing device 10 acquires the environmental conditions surrounding the mobile body when it previously traveled a predetermined route. For example, the control unit 15 acquires the environmental conditions surrounding the mobile body, which are stored in the storage unit 12 as a record of the mobile body's past movements, by referring to the storage unit 12. This recording of environmental conditions may be based on past input operations by the user using the input unit 13 of the information processing device 10, or it may be based on past reception operations from an external server device of the information processing system 1 using the communication unit 11 of the information processing device 10. For example, the control unit 15 may acquire environmental conditions that match the past movement route and date and time of movement of the mobile body loaded with cargo as information from the server device via the communication unit 11.

[0047] In step S203, the control unit 15 of the information processing device 10 acquires the loading status of the cargo when the mobile body carrying the cargo has previously traveled a predetermined route. For example, the control unit 15 acquires the loading status of the cargo stored in the storage unit 12 as a record of the past movement of the mobile body carrying the cargo by referring to the storage unit 12. Such recording of the loading status may be based on past input operations by the user using the input unit 13 of the information processing device 10, or it may be based on past reception operations from the imaging device 20 using the communication unit 11 of the information processing device 10.

[0048] In step S204, the control unit 15 of the information processing device 10 stores in the storage unit 12 past measured data, which associates the movement conditions of the moving object, at least one of the environmental conditions around the moving object, and the loading state of the cargo for each movement route, based on the information acquired in steps S201 to S203.

[0049] In step S205, the control unit 15 of the information processing device 10 constructs a predictive model based on past measured data stored in the storage unit 12 in step S204. For example, the control unit 15 learns the loading state of the cargo according to at least one of the movement conditions of the moving object and the environmental conditions surrounding the moving object, based on past measured data, and constructs a predictive model. The control unit 15 constructs the predictive model within the information processing device 10 itself and acquires the predictive model within the device. The control unit 15 stores the constructed predictive model in the storage unit 12.

[0050] A predictive model is a machine learning model that has been trained based on historically acquired real-world data. For example, a predictive model is a supervised learning model that takes at least one of the movement conditions of a moving object and the surrounding environmental conditions of the moving object as input data, and the loading state of the cargo as training data, and is a mathematical model that learns the loading state of the cargo according to at least one of the movement conditions of the moving object and the surrounding environmental conditions. A supervised learning model is, for example, a neural network that includes an input layer, one or more hidden layers, and an output layer, but is not limited to this.

[0051] The training of the supervised learning model is performed by the control unit 15 of the information processing device 10. The training of the supervised learning model may be batch learning or online learning. In this disclosure, "constructing a predictive model" means a state in which batch learning is completed or online learning has been performed to a certain extent. In the case of online learning, learning may continue even after learning has been performed to a certain extent.

[0052] In the following section, with reference to Figures 4 to 6, we will mainly describe the process by which the control unit 15 of the information processing device 10 notifies notification information using the prediction model constructed in step S205 of Figure 3.

[0053] Figure 4 is a flowchart illustrating a third example of the information processing method performed by the information processing device 10 in Figure 1. The third example of the information processing method performed by the information processing device 10 in Figure 1 will be mainly explained with reference to Figure 4.

[0054] In step S301, the control unit 15 of the information processing device 10 inputs, for example, the history of movement conditions immediately preceding the movement of the moving object and at least one of the environmental conditions on the movement route to which the moving object is scheduled to travel, into the prediction model constructed in step S205 of Figure 3.

[0055] In step S302, the control unit 15 of the information processing device 10 predicts the loading status of the cargo along the travel route that the moving object is scheduled to travel, based on the prediction model that was input in step S301.

[0056] In step S303, the control unit 15 of the information processing device 10 notifies the loading status of the cargo predicted in step S302 as first predicted information included in the notification information.

[0057] Figure 5 is a flowchart illustrating a fourth example of the information processing method performed by the information processing device 10 in Figure 1. The fourth example of the information processing method performed by the information processing device 10 in Figure 1 will be mainly explained with reference to Figure 5.

[0058] In step S401, the control unit 15 of the information processing device 10 periodically or irregularly acquires images of the cargo captured by the imaging device 20 from the imaging device 20 via the communication unit 11. The control unit 15 acquires multiple images of the cargo, which are repeatedly captured using the imaging device 20 while the moving object is in motion, in a time series.

[0059] In step S402, the control unit 15 of the information processing device 10 analyzes the multiple images acquired in step S401 based on the same image processing as in step S102 in Figure 2. By analyzing the images, the control unit 15 acquires information about the loading status of the cargo on the moving object. For example, the control unit 15 acquires information about the loading status of steel coils by analyzing images taken inside the cargo hold of a ship.

[0060] In step S403, the control unit 15 of the information processing device 10 determines whether or not cargo displacement has occurred based on the loading state of the cargo obtained from the image analysis in step S402. If the control unit 15 determines that cargo displacement has occurred, it executes the process in step S404. If the control unit 15 determines that cargo displacement has not occurred, it terminates the process.

[0061] In step S404, if the control unit 15 of the information processing device 10 determines that cargo displacement has occurred in step S403, it predicts a movement route that suppresses cargo collapse due to the movement of the moving object, based on the prediction model constructed in step S205 in Figure 3.

[0062] In step S405, the control unit 15 of the information processing device 10 notifies the future movement route of the moving object predicted in step S404 as second prediction information included in the notification information.

[0063] Figure 6 is a flowchart illustrating a fifth example of the information processing method performed by the information processing device 10 in Figure 1. The fifth example of the information processing method performed by the information processing device 10 in Figure 1 will be mainly explained with reference to Figure 6.

[0064] In step S501, the control unit 15 of the information processing device 10 periodically or irregularly acquires images of the cargo captured by the imaging device 20 from the imaging device 20 via the communication unit 11. The control unit 15 acquires multiple images of the cargo, which are repeatedly captured using the imaging device 20 while the moving object is in motion, in a time series.

[0065] In step S502, the control unit 15 of the information processing device 10 analyzes the multiple images acquired in step S501 based on the same image processing as in step S102 in Figure 2. By analyzing the images, the control unit 15 acquires information about the loading status of the cargo on the moving object. For example, the control unit 15 acquires information about the loading status of steel coils by analyzing images taken inside the cargo hold of a ship.

[0066] In step S503, the control unit 15 of the information processing device 10 determines whether or not cargo displacement has occurred based on the loading state of the cargo obtained from the image analysis in step S502. If the control unit 15 determines that cargo displacement has occurred, it executes the process in step S504. If the control unit 15 determines that cargo displacement has not occurred, it terminates the process.

[0067] In step S504, if the control unit 15 of the information processing device 10 determines that cargo displacement has occurred in step S503, it predicts at least one of the movement conditions and environmental conditions that suppress cargo collapse due to the movement of the moving object, based on the prediction model constructed in step S205 of Figure 3.

[0068] In step S505, the control unit 15 of the information processing device 10 notifies the third predicted information included in the notification information, which is at least one of the movement conditions and environmental conditions predicted in step S504.

[0069] Figure 7 is the first diagram illustrating an example of processing by the information processing device 10 in Figure 1. Figure 7 shows an example of past measured data stored in the storage unit 12 in step S204 of Figure 3.

[0070] As shown in Figure 7, past measurement data includes information about predetermined travel routes that a mobile body loaded with cargo has actually traveled in the past. In past measurement data, each of these travel routes is associated with information about the past travel conditions of the mobile body when it traveled along the corresponding route. In past measurement data, each of these travel routes is associated with information about the past surrounding environmental conditions of the mobile body when it traveled along the corresponding route. In past measurement data, each of these travel routes is associated with information about the past loading state of the cargo when the mobile body traveled along the corresponding route.

[0071] Figure 8 is a second diagram illustrating an example of processing by the information processing device 10 in Figure 1. Figure 8 shows an example of the arrangement of the imaging device 20 for capturing images acquired by the control unit 15 of the information processing device 10 in step S101 of Figure 2.

[0072] The imaging device 20 is installed in the cargo hold, for example, to monitor the loading status of steel coils stored in the cargo hold. A pair of imaging devices 20 are installed on the ceiling side of the cargo hold, facing each other, to image the entire interior of the cargo hold.

[0073] Each control unit 24 of the pair of imaging devices 20 uses an imaging unit 23 to image the steel coils loaded in the cargo hold from the ceiling side of the cargo hold and acquires an image. Each control unit 24 of the pair of imaging devices 20 stores the image of the steel coils loaded in the cargo hold in a storage unit 22 and transmits it via a communication unit 21 to an information processing device 10 installed in the wheelhouse or other location of the ship having the cargo hold. The control unit 15 of the information processing device 10 receives the image of the steel coils loaded in the cargo hold from each of the pair of imaging devices 20 via a communication unit 11.

[0074] Figure 9 is the third figure illustrating an example of processing by the information processing device 10 in Figure 1. Figure 10 is the fourth figure illustrating an example of processing by the information processing device 10 in Figure 1. Figures 9 and 10 show an example of load displacement of steel coils related to image analysis performed by the control unit 15 of the information processing device 10 in step S102 of Figure 2, etc.

[0075] For example, steel coils loaded in a ship's hold may shift as the ship moves. For instance, steel coil C3 loaded on top of steel coils C1 and C2 may rotate or translate as the ship moves and is subjected to tilting or impact from waves, as shown in Figure 9. When steel coil C3 causes shifting due to rotation and translation, steel coils C1 and C2 on which steel coil C3 is placed may tilt relative to their axial direction, as shown in Figure 10.

[0076] The control unit 15 of the information processing device 10 acquires information on the loading status of steel coils by analyzing images captured using the imaging device 20 in the ship's cargo hold. The control unit 15 performs image processing on each image acquired from the imaging device 20 via the communication unit 11, using a Sobel filter that extracts contours from the image using the first derivative. The control unit 15 acquires contour images for each image acquired from the imaging device 20 via the communication unit 11. The control unit 15 then superimposes the acquired contour images.

[0077] In this case, as shown in Figures 9 and 10, if the loading state of steel coils C1, C2, and C3 changes and load shifting occurs, the contours of each steel coil will not overlap between one image and the other image. For example, if steel coil C3 is rotating and translating, the contour of steel coil C3 will not overlap between one image and the other image. For example, if each of steel coils C1 and C2 is tilted with respect to the axial direction, the contours of each of steel coils C1 and C2 will not overlap between one image and the other image.

[0078] The control unit 15 identifies steel coils that are experiencing load displacement by determining whether the contours of one image and another image match or not. For example, the control unit 15 identifies steel coil C3 that is experiencing load displacement based on rotation and translational movement. For example, the control unit 15 identifies steel coils C1 and C2 that are experiencing load displacement based on inclination with respect to the axial direction.

[0079] According to the information processing device 10 of the above embodiment, it is possible to suppress cargo collapse of a load loaded on a moving vehicle. The information processing device 10 notifies the user of notification information that supports the suppression of cargo collapse accompanying the movement of the moving vehicle, based on at least one of the moving conditions of the moving vehicle and the surrounding environmental conditions of the moving vehicle, and the loading state of the load. As a result, when predetermined conditions are met, the information processing device 10 can prompt the user to take necessary measures to suppress cargo collapse accompanying the movement of the moving vehicle. Based on the notification information received from the information processing device 10, the user can take necessary measures to prevent cargo collapse accompanying the movement of the moving vehicle. As a result, cargo collapse of a load loaded on a moving vehicle can be suppressed.

[0080] When the information processing device 10 determines that cargo displacement has occurred based on the loading status of the cargo, it notifies the user of alert information as notification information. This allows the information processing device 10 to notify the user as a fact that cargo displacement has occurred due to the movement of the moving object. The user can easily understand as a fact that cargo displacement has occurred, for example, based on the alert information included in the notification information received from the information processing device 10. This allows the user to, for example, change the moving route of the moving object or appropriately change the moving conditions of the moving object, such as acceleration and speed, when cargo displacement occurs during the movement of the moving object, in order to avoid further cargo displacement and collapse.

[0081] When the information processing device 10 determines that the history of the movement conditions immediately preceding the movement of the moving object satisfies the first condition, it notifies the user of alert information. This allows the information processing device 10 to notify the user that there is a high probability that cargo displacement is occurring as the moving object moves. The user can easily understand that there is a high probability that cargo displacement is occurring, for example, based on the alert information included in the notification information received from the information processing device 10. This allows the user to, for example, change the movement route of the moving object or appropriately change the movement conditions of the moving object, such as acceleration and speed, at the moment they recognize that there is a high probability that cargo displacement is occurring during the movement of the moving object, in order to avoid further cargo displacement and collapse.

[0082] When the information processing device 10 determines that the environmental conditions along the planned route of the moving object meet the second condition, it notifies the user of alert information. This allows the information processing device 10 to notify the user that there is a high probability that cargo displacement is occurring as the moving object moves. The user can easily understand that there is a high probability that cargo displacement is occurring, for example, based on the alert information included in the notification received from the information processing device 10. This allows the user to, for example, change the moving route of the moving object or appropriately change the moving conditions of the moving object, such as acceleration and speed, at the moment they recognize that there is a high probability that cargo displacement is occurring during the movement of the moving object, in order to avoid further cargo displacement and collapse.

[0083] The information processing device 10 uses a predictive model based on past measured data that correlates the movement conditions of the moving object, at least one of the environmental conditions surrounding the moving object, and the loading state of the cargo for each movement route, and notifies the user of the predictive information as notification information. This allows the information processing device 10 to prompt the user to take more accurate measures necessary to suppress cargo collapse as the moving object moves. Based on the predictive information received from the information processing device 10, the user can take more accurate measures to prevent cargo collapse as the moving object moves.

[0084] The information processing device 10 predicts the loading status of the cargo along the planned travel route of the moving object based on a prediction model, and notifies the user of the predicted loading status as first prediction information. This allows the information processing device 10 to predict, for example, the loading status of the cargo along the planned travel route of the moving object, and prompt the user to take appropriate action.

[0085] For example, if the information processing device 10 predicts that continuing to move the moving object along its current route will cause the cargo to shift, the user can change the route at that point. Alternatively, if the information processing device 10 predicts that continuing to move the moving object along its current route will not cause the cargo to shift, the user can maintain the current route. Or, the user can indirectly determine the optimal route that will prevent cargo shifting based on the first prediction information provided by the information processing device 10, even before the moving object begins to move.

[0086] The information processing device 10 predicts a movement route that suppresses cargo collapse as the moving object moves, based on a prediction model, and notifies the user of the predicted movement route as second prediction information. As a result, the information processing device 10 can, for example, predict a movement route that suppresses cargo collapse as the moving object moves, and prompt the user to take appropriate action.

[0087] For example, if the user detects cargo displacement based on images from the imaging device 20 while the moving object is in motion, the user can change the moving route to the optimal route based on the second prediction information at that point. Even if the user has not detected cargo displacement based on images from the imaging device 20 while the moving object is in motion, the user can change the moving route in advance to the optimal route based on the second prediction information before cargo displacement occurs. Alternatively, the user can directly determine the optimal moving route that will prevent cargo collapse based on the second prediction information notified by the information processing device 10, even before the moving object starts moving.

[0088] The information processing device 10 predicts at least one of the movement conditions and environmental conditions that suppress cargo collapse due to the movement of a moving object, based on a prediction model, and notifies the user of at least one of the predicted movement conditions and environmental conditions as third prediction information. As a result, the information processing device 10 can, for example, predict at least one of the movement conditions and environmental conditions that suppress cargo collapse due to the movement of a moving object, and prompt the user to take appropriate action.

[0089] For example, if the user detects cargo displacement based on images from the imaging device 20 while the moving object is in motion, the user can change the moving conditions of the moving object to the optimal moving conditions based on the third predictive information at that point. Even if the user has not detected cargo displacement based on images from the imaging device 20 while the moving object is in motion, the user can change the moving conditions of the moving object in advance to the optimal moving conditions based on the third predictive information before cargo displacement occurs. Alternatively, the user can directly determine the optimal moving conditions that will prevent cargo collapse based on the third predictive information notified by the information processing device 10, even before the moving object starts moving.

[0090] For example, if the user detects cargo displacement based on images from the imaging device 20 while the mobile body is moving, the user can change the mobile body's route to achieve optimal environmental conditions based on the third predictive information at that time. Even if the user has not detected cargo displacement based on images from the imaging device 20 while the mobile body is moving, the user can change the mobile body's route in advance to achieve optimal environmental conditions based on the third predictive information before cargo displacement occurs. Alternatively, the user can indirectly determine the optimal route that will result in environmental conditions that prevent cargo collapse, based on the third predictive information notified by the information processing device 10, before the mobile body even moves.

[0091] The information processing device 10 can suppress cargo collapse of steel coils and other cargo loaded on a ship by including the ship as the moving object. Based on the alert information and predictive information described above, the information processing device 10 can prompt the user to take necessary actions to suppress cargo collapse due to the movement of the ship. Based on this notification information received from the information processing device 10, the user can take necessary actions to prevent cargo collapse due to the movement of the ship. As a result, cargo collapse is suppressed.

[0092] The information processing device 10 acquires information about the loading status of cargo by analyzing images taken inside the ship's cargo hold. As a result, the information processing device 10 can accurately detect cargo displacement and other issues based on images of the cargo inside the cargo hold taken by the imaging device 20.

[0093] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions included in each configuration or step can be rearranged in a logically consistent manner, and multiple configurations or steps can be combined into one or divided.

[0094] For example, it is also possible to configure a general-purpose electronic device such as a smartphone or computer to function as the information processing device 10 according to the above-described embodiment. Specifically, a program describing the processing content that realizes each function of the information processing device 10 according to the embodiment is stored in the memory of the electronic device, and the processor of the electronic device reads and executes the program. Therefore, this disclosure can also be implemented as a program that can be executed by a processor.

[0095] Alternatively, the disclosure may also be implemented as a non-temporary computer-readable medium storing a program executable by one or more processors for causing an information processing device 10 or the like to perform each function according to one embodiment. It should be understood that these are also included within the scope of the disclosure.

[0096] For example, at least some of the processing operations performed in the information processing device 10 in the above-described embodiment may be performed in the imaging device 20. At least some of the processing operations performed in the imaging device 20 may be performed in the information processing device 10.

[0097] In the above embodiment, the information processing device 10 is described as notifying the user of alert information to help suppress cargo collapse due to the movement of the moving object when all conditions are met in the determination process of steps S103 to S105 in Figure 2, but it is not limited to this. The information processing device 10 may also notify the user of alert information to help suppress cargo collapse due to the movement of the moving object when at least one condition is met in the determination process of steps S103 to S105 in Figure 2.

[0098] The information processing device 10 may arbitrarily determine conditions, such as if-then rules, for notifying the user of alert information, without being limited to the conditions shown in the flowchart of Figure 2. For example, such conditions may be objectively determined based on past measured data relating at least one of the movement conditions of the moving object and the environmental conditions surrounding the moving object with the loading state of the cargo for each movement route, or they may be subjectively determined based on the user's rules of thumb.

[0099] In the embodiments described above, the predictive model includes, but is not limited to, a machine learning model constructed using a neural network. The predictive model may also include a machine learning model constructed using other methods, such as multiple regression analysis.

[0100] In the above embodiment, the information processing device 10 was described as performing calculations related to the construction of a prediction model on its own and constructing the prediction model, but it is not limited to this. The information processing device 10 may also acquire a prediction model by receiving an already constructed prediction model from any other external device via the network and communication unit 11.

[0101] In the above embodiment, the information processing device 10 was described as notifying the user of notification information including alert information and prediction information, but it is not limited to this. The information processing device 10 may notify the user of only one of either the alert information or the prediction information as notification information.

[0102] In the above embodiment, the information processing device 10 was described as notifying the user of prediction information including the first prediction information, the second prediction information, and the third prediction information as notification information, but it is not limited to this. The information processing device 10 may notify the user of any one or any two of the first prediction information, the second prediction information, and the third prediction information as notification information.

[0103] In the above embodiments, the moving body was described as including ships, but is not limited to them. The moving body is not limited to those that move on the sea, but may also include any other things that move on land, in the air, etc. For example, the moving body may include vehicles and railways that move on land, or aircraft that move in the air.

[0104] In the above embodiment, the information processing device 10 was described as acquiring information on the loading state by analyzing images taken inside the ship's cargo hold, but it is not limited to this. The information processing device 10 may acquire information on the loading state of the cargo based on any other method different from the image analysis method. For example, the information processing device 10 may acquire sensor information based on other sensor devices other than the imaging device 20 installed inside the cargo hold, and acquire information on the loading state by analyzing the acquired sensor information.

[0105] In the above embodiment, the information processing system 1 was described as having an information processing device 10 and an imaging device 20, but it is not limited to this. In addition to the information processing device 10 and the imaging device 20, the information processing system 1 may further have a server device installed outside the mobile body. At least some of the processing operations performed in the information processing device 10 in the above embodiment may be performed in the server device.

[0106] The information processing device 10, the imaging device 20, and the server device are each connected to a network, including a mobile communication network and the Internet, in a manner that enables communication. In this case, the communication unit of each device includes one or more communication interfaces that connect to the network. The communication interface may support, for example, mobile communication standards such as 4G (4th Generation) and 5G (5th Generation), wired LAN (Local Area Network) standards, or wireless LAN standards, but is not limited to these and may support any communication standard. For example, the communication interface may also support short-range wireless communication standards. The information processing device 10, the imaging device 20, and the server device are connected to each other in a manner that enables communication via the network.

[0107] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] A control unit that notifies the user of notification information to help suppress cargo collapse during the movement of a mobile body loaded with cargo, based on at least one of the movement conditions of the mobile body and the surrounding environmental conditions of the mobile body, and the state of the cargo loading. Equipped with, Information processing device. [Note 2] The information processing device described in Appendix 1, When the control unit determines, based on the loading state, that a shift in the cargo has occurred, it notifies alert information as notification information. Information processing device. [Note 3] An information processing device as described in Appendix 1 or 2, When the control unit determines that the history of the movement conditions associated with the movement of the moving object up to the immediate prior to the first condition has been met, it notifies the alert information as the notification information. Information processing device. [Note 4] An information processing device described in any one of the appendices 1 to 3, When the control unit determines that the environmental conditions along the route on which the moving object is scheduled to travel satisfy the second condition, it notifies the alert information as the notification information. Information processing device. [Note 5] An information processing device described in any one of the appendices 1 to 4, The control unit uses a prediction model based on past measured data that correlates the movement conditions of the moving body and at least one of the environmental conditions surrounding the moving body with the loading state of the cargo for each movement route, and notifies the prediction information as notification information. Information processing device. [Note 6] The information processing device described in Appendix 5, The control unit predicts the loading state along the planned route of the moving object based on the prediction model, and notifies the predicted loading state as first prediction information. Information processing device. [Note 7] An information processing device as described in Appendix 5 or 6, The control unit predicts the movement route that suppresses cargo collapse as the moving body moves, based on the prediction model, and notifies the predicted movement route as second prediction information. Information processing device. [Note 8] An information processing device described in any one of the appendices 5 to 7, The control unit predicts, based on the prediction model, at least one of the movement conditions and environmental conditions that suppress cargo collapse due to the movement of the moving body, and notifies the predicted at least one of the movement conditions and environmental conditions as third prediction information. Information processing device. [Note 9] An information processing device described in any one of the appendices 1 to 8, The aforementioned moving body includes a ship, Information processing device. [Note 10] The information processing device described in Appendix 9, The control unit acquires information about the loading state by analyzing images taken inside the cargo hold of the ship. Information processing device. [Note 11] An information processing device described in any one of the appendices 1 to 10, An imaging device that captures the loading state and transmits the image obtained to the information processing device, Equipped with, Information processing system. [Note 12] When a mobile body loaded with cargo is in motion, notification information is provided to the user to help suppress cargo collapse during the movement of the mobile body, based on at least one of the conditions under which the mobile body is moving, the environmental conditions surrounding the mobile body, and the state of the cargo loading. including, Information processing methods. [Note 13] In an information processing device, When a mobile body loaded with cargo is in motion, notification information is provided to the user to help suppress cargo collapse during the movement of the mobile body, based on at least one of the conditions under which the mobile body is moving, the environmental conditions surrounding the mobile body, and the state of the cargo loading. Perform an action that includes program. [Explanation of Symbols]

[0108] 1. Information Processing System 10 Information Processing Devices 11 Communications Department 12 Storage section 13 Input section 14 Output section 15 Control Unit 20 Imaging device 21 Communications Department 22 Memory section 23 Imaging Unit 24 Control Unit C1, C2, C3 Steel Coil

Claims

1. A control unit that notifies the user of notification information to help suppress cargo collapse during the movement of a mobile body loaded with cargo, based on at least one of the movement conditions of the mobile body and the surrounding environmental conditions of the mobile body, and the state of the cargo loading. Equipped with, The control unit uses a prediction model based on past measured data that correlates the movement conditions of the moving body and at least one of the environmental conditions surrounding the moving body with the loading state of the cargo for each movement route, and notifies the prediction information as notification information. Information processing device.

2. An information processing apparatus according to claim 1, When the control unit determines, based on the loading state, that a shift in the cargo has occurred, it notifies alert information as notification information. Information processing device.

3. An information processing apparatus according to claim 1 or 2, When the control unit determines that the history of the movement conditions associated with the movement of the moving body up to the immediate prior to the first condition has been met, it notifies the alert information as the notification information. Information processing device.

4. An information processing apparatus according to claim 1 or 2, When the control unit determines that the environmental conditions along the route on which the moving object is scheduled to travel satisfy the second condition, it notifies the alert information as the notification information. Information processing device.

5. An information processing apparatus according to claim 1, The control unit predicts the loading state along the planned route of the moving object based on the prediction model, and notifies the predicted loading state as first prediction information. Information processing device.

6. An information processing apparatus according to claim 1, The control unit predicts the movement route that suppresses cargo collapse as the moving body moves, based on the prediction model, and notifies the predicted movement route as second prediction information. Information processing device.

7. An information processing apparatus according to claim 1, The control unit predicts, based on the prediction model, at least one of the movement conditions and environmental conditions that suppress cargo collapse due to the movement of the moving body, and notifies the predicted at least one of the movement conditions and environmental conditions as third prediction information. Information processing device.

8. An information processing apparatus according to claim 1 or 2, The aforementioned moving body includes a ship, Information processing device.

9. An information processing apparatus according to claim 8, The control unit acquires information about the loading state by analyzing images taken inside the cargo hold of the ship. Information processing device.

10. The information processing apparatus according to claim 1 or 2, An imaging device that captures the loading state and transmits the image obtained to the information processing device, Equipped with, Information processing system.

11. When a mobile body loaded with cargo is in motion, notification information is provided to the user to help suppress cargo collapse during the movement of the mobile body, based on at least one of the conditions under which the mobile body is moving, the environmental conditions surrounding the mobile body, and the state of the cargo loading. Includes, Notifying the user of the aforementioned notification information includes notifying the user of the prediction information as the aforementioned notification information, using a prediction model based on past measured data that correlates at least one of the movement conditions of the moving body and the environmental conditions surrounding the moving body with the loading state of the cargo for each movement route. Information processing methods.

12. In an information processing device, When a mobile body loaded with cargo is in motion, notification information is provided to the user to help suppress cargo collapse during the movement of the mobile body, based on at least one of the conditions under which the mobile body is moving, the environmental conditions surrounding the mobile body, and the state of the cargo loading. A program for performing an action that includes the following: The operation includes using a predictive model based on past measured data that correlates the movement conditions of the moving body and at least one of the environmental conditions surrounding the moving body with the loading state of the cargo for each movement route, and notifying the predictive information as the notification information. program.

Images