Control device, load-carrying platform monitoring system, and notification method

Abstract

To provide a control device capable of allowing a user to easily recognize a change in a detection object in a load-carrying platform.SOLUTION: The control device is a control device used for a vehicle and includes: an image acquisition part 10 for acquiring a load-carrying platform image Gb captured by an imaging device 5 for photographing a load-carrying platform of the vehicle; a load-carrying platform detection part 12 for detecting a change in a detection object in the load-carrying platform from the load-carrying platform image Gb; and a notification processing part 13 for notifying a user of the vehicle of the change in the detection object by at least one of sound output by a voice output part 4 and display of a notification image by an image output part 3, when the load-carrying platform detection part 12 detects the change in the detection object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosure herein relates to a control device, a loading platform monitoring system, and a notification method. [Background technology]

[0002] Patent Document 1 discloses a cargo shift monitoring device that photographs the inside of a loading platform and displays the photographed image on a display device in the driver's seat. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-18684 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the captured image is simply continuously displayed on a display device in the driver's seat, and no changes in the loading platform are detected and notified, making it difficult for the driver to notice changes in the loading platform.

[0005] One disclosed object is to provide a control device that allows the user to easily recognize changes in the detection target within the cargo bed. [Means for solving the problem]

[0006] The control device disclosed herein is a control device used in a vehicle (6), and includes an image acquisition unit (10) that acquires a loading platform image (Gb) captured by an imaging device (5) that captures an image of the loading platform (6b) of the vehicle; a loading platform detection unit (12) that detects a change in a detection object (Ob) within the loading platform from an image of the loading platform; The vehicle is provided with a notification processing unit (13) that, when the loading platform detection unit detects a change in the detection target, notifies the user of the change in the detection target by at least one of outputting sound from the audio output unit (4) and displaying a notification image (Pi) from the image output unit (3).

[0007] The disclosed loading platform monitoring system is a loading platform monitoring system for use in a vehicle (6), The system includes an imaging device (5) for photographing a loading platform (6b) of a vehicle and a control device (1), The control device an image acquisition unit (10) that acquires a loading platform image (Gb) captured by an imaging device; a loading platform detection unit (12) that detects a change in a detection object (Ob) within the loading platform from an image of the loading platform; The vehicle has a notification processing unit (13) that, when the loading platform detection unit detects a change in the detection target, notifies the user of the change in the detection target by at least one of outputting sound from the audio output unit (4) and displaying a notification image (Pi) from the image output unit (3).

[0008] Also disclosed is a notification method for use in a vehicle (6), comprising: An image of the loading platform (Gb) of the vehicle is captured by an imaging device (5) for capturing an image of the loading platform (6b) of the vehicle (S11). A change in the detection object (Ob) in the loading platform is detected from the loading platform image (S12). The notification method includes, in the processing executed by at least one processor (14), a step of notifying a user of the vehicle of the change in the detection target by at least one of outputting sound from an audio output unit (4) and displaying a notification image (Pi) from an image output unit (3) when a change in the detection target is detected (S13).

[0009] According to these, when a change in the detection target in the loading platform is detected from a loading platform image taken of the loading platform, the change in the detection target is notified to the user by at least one of sound output and display of a notification image. In this way, by notifying the user using the change in the detection target as a trigger, the user can easily recognize the change in the detection target in the loading platform. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 illustrates a loading platform monitoring system. [Figure 2] FIG. [Figure 3] FIG. [Figure 4] FIG. 10 is a diagram illustrating protrusion detection. [Figure 5] FIG. 10 is a diagram illustrating protrusion detection. [Figure 6] FIG. 10 is a diagram illustrating protrusion detection. [Figure 7] FIG. 10 is a diagram illustrating human detection. [Figure 8] FIG. 10 is a diagram illustrating sway detection. [Figure 9] FIG. 10 is a diagram illustrating an example of a notification image. [Figure 10] FIG. 10 is a diagram illustrating an example of a notification image. [Figure 11] FIG. 10 is a diagram illustrating an example of a notification image. [Figure 12] FIG. 10 is a diagram illustrating an example of a notification image. [Figure 13] FIG. 10 is a diagram illustrating an example of a notification image. [Figure 14] FIG. 10 is a diagram illustrating an example of a notification image. [Figure 15] 4 is a flowchart showing processing in a periphery monitoring ECU. [Figure 16] 4 is a flowchart showing processing in a periphery monitoring ECU. [Figure 17] 4 is a flowchart showing processing in a periphery monitoring ECU. [Figure 18] 4 is a flowchart showing processing in a periphery monitoring ECU. [Figure 19] 4 is a flowchart showing processing in a periphery monitoring ECU. [Figure 20] 4 is a flowchart showing processing in a periphery monitoring ECU. DETAILED DESCRIPTION OF THE INVENTION

[0011] The bed monitoring system 100 shown in FIG. 1 is mounted on a vehicle 6. The bed monitoring system 100 monitors the bed 6b of the vehicle 6, and notifies the user of the vehicle 6 when it detects a change in the detection object Ob. The vehicle 6 is a cargo vehicle (e.g., a pickup truck) with an open top of the bed 6b. The bed monitoring system 100 includes a state detection unit 2, a display unit 3, an audio output unit 4, an imaging unit 5, and a periphery monitoring ECU 1. The minimum components included in the bed monitoring system 100 are the periphery monitoring ECU 1 and the imaging unit 5.

[0012] In this disclosure, the concept of a right-handed three-dimensional coordinate system having mutually orthogonal X-, Y-, and Z-axes will be introduced for explanation (see FIG. 2). Specifically, the traveling direction of the vehicle 6 is defined as the forward direction, and the direction opposite to the forward direction is defined as the backward direction. The Y-axis is defined along the longitudinal direction. The X-axis is defined along the vehicle width direction. The Z-axis is defined along the vertical direction of the vehicle 6. The downward direction is also the direction of gravity.

[0013] The state detection unit 2 is a sensor that detects the state of the vehicle 6. The state detection unit 2 includes a brake sensor, an accelerator sensor, a shift position sensor 20, a vehicle speed sensor, an acceleration sensor, etc. The shift position sensor 20 is a sensor that detects the position of the shift lever. Each sensor outputs data indicating the current value (i.e., the detection result) of the physical state quantity to be detected to the LAN. The output data of each sensor is acquired by the periphery monitoring ECU 1 or the like via the LAN. Note that the types of sensors used by the periphery monitoring ECU 1 as the state detection unit 2 may be designed as appropriate, and it is not necessary to include all of the sensors described above.

[0014] The display unit 3 is an image output unit that displays a predetermined image to the driver of the vehicle 6. The display unit 3 is a liquid crystal display, an organic EL display, or the like. The display unit 3 is disposed in front of the driver's seat in the passenger compartment of the vehicle 6, and is fixed to a fixed member (for example, an instrument panel, etc.) on the vehicle 6 side with the display screen facing the driver's seat. The display unit 3 notifies the driver of the vehicle 6 of predetermined information by displaying an image. The display unit 3 outputs visual information to the driver based on image output data input from the periphery monitoring ECU 1. The display output data is data related to a notification image Pi output by a notification processing unit 13, which will be described later.

[0015] The display unit 3 has a touch panel function. The display unit 3 detects, for example, a touch operation, a swipe operation, etc. on the display screen by the user. The display unit 3 transmits operation information input by the user to the periphery monitoring ECU 1.

[0016] The audio output unit 4 is an in-vehicle speaker. The audio output unit 4 notifies the driver of the vehicle 6 of predetermined information by sound. The audio output unit 4 notifies the user by sound based on sound output data input from the periphery monitoring ECU 1. The sound output data is data output by a notification processing unit 13, which will be described later.

[0017] The imaging unit 5 is an imaging device that captures an image of the loading platform 6b of the vehicle 6. As shown in FIG. 2, the imaging unit 5 is disposed on the upper rear side of the cabin of the vehicle 6. The imaging unit 5 is also called a bed camera. The number and arrangement of the imaging units 5 may be arbitrary.

[0018] The imaging unit 5 captures an image of the loading platform 6b at predetermined time intervals. The imaging unit 5 transmits a loading platform image Gb to the periphery monitoring ECU 1. The loading platform image Gb is an image of the loading platform 6b captured by the imaging unit 5. The timing at which the imaging unit 5 transmits the loading platform image Gb may be arbitrary.

[0019] The imaging unit 5 captures an image of the loading platform 6b when the vehicle 6 is stopped. When the vehicle is stopped, it means that it detects that the shift lever is in the parking position (hereinafter, P position). Alternatively, when the vehicle is stopped, it may mean that it detects that the shift lever is in the P position and the parking brake is on. The imaging unit 5 transmits the loading platform image Gb when the vehicle 6 is stopped to the periphery monitoring ECU 1 as a stopped vehicle image Gs (see FIG. 1).

[0020] The imaging unit 5 captures images of the loading platform 6b at predetermined time intervals while the vehicle 6 is stopped. The imaging unit 5 transmits the loading platform images Gb captured while the vehicle 6 is stopped to the periphery monitoring ECU 1 as stopped vehicle images Gs.

[0021] The imaging unit 5 captures an image of the cargo bed 6b when the vehicle 6 restarts. The restart refers to when the shift lever is changed from the P setting to a setting other than the P setting after the vehicle 6 has stopped. The restart may also refer to when it is detected that the ignition switch has been turned on by the user. The imaging unit 5 transmits the cargo bed image Gb taken when the vehicle 6 restarts to the periphery monitoring ECU 1 as a restart image Gr (see FIG. 1).

[0022] The periphery monitoring ECU 1 is mainly composed of a microcomputer having a processor 14, memory, I / O, and a bus connecting these. The periphery monitoring ECU 1 corresponds to a control device. The periphery monitoring ECU 1 performs various processes by having the processor 14 execute a control program stored in the memory. As shown in FIG. 15, the periphery monitoring ECU 1 acquires a bed image Gb (S11), and when it detects a change in the detection object Ob from the bed image Gb (S12), it notifies the user of the vehicle 6 (S13). The execution of the processes from S11 to S13 by the processor 14 corresponds to the execution of a notification method.

[0023] As shown in FIG. 1, the periphery monitoring ECU 1 includes an image acquisition unit 10, an area setting unit 11, a loading platform detection unit 12, and a notification processing unit 13. The image acquisition unit 10 acquires a loading platform image Gb, a stopped vehicle image Gs, and a restart image Gr from the imaging unit 5. The image acquisition unit 10 records the acquired images in memory. The image acquisition unit 10 records the first stopped vehicle image Gs acquired immediately after the vehicle 6 stops as an immediately after stopping image. The immediately after stopping image is the stopped vehicle image Gs immediately after the vehicle stops. The immediately after stopping image is updated when the vehicle 6 restarts and is stopped again.

[0024] The area setting unit 11 sets a loading area area E, which is an area within the loading area 6b, based on operation information of an operation by the user. The loading area area E is an area set to detect whether the detection object Ob protrudes from the loading area 6b. The loading area area E is set by the user operating the touch panel when the periphery monitoring ECU 1 is initially set. The time when the periphery monitoring ECU 1 is initially set is when the periphery monitoring ECU 1 is first started, for example. The loading area area E may be set at a time other than the time of initial setting. As shown in FIG. 2 or FIG. 3, the loading area area E is set as a surface having a predetermined range on the XY plane.

[0025] <Initial settings for the cargo area> The area setting unit 11 acquires an initial image Gi, which is a loading platform image Gb in a state where nothing is placed on the loading platform 6b. The initial image Gi is captured when the vehicle 6 is shipped from the factory and is recorded in advance in memory. The periphery monitoring ECU 1 may extract a loading platform image Gb in a state where nothing is placed on the loading platform 6b from the acquired loading platform images Gb, and set the extracted loading platform image Gb as the initial image Gi. The determination of whether or not nothing is placed on the loading platform 6b is made using image recognition technology. The area setting unit 11 controls the display unit 3 to display the initial image Gi. The area setting unit 11 prompts the user to input the four corner points C that define the loading platform area E. The area setting unit 11 sets the loading platform area E based on the four corner points C input by the user.

[0026] The loading platform area E is a surface bounded by four sides Ex1, Ex2, Ey1, and Ey2 that connect the four corner points C. Side Ex1 connects two points C1 and C2 that are located in the negative Y-axis direction (front edge of the loading platform) among the points C. Side Ex2 connects two points C3 and C4 that are located in the positive Y-axis direction (rear edge of the loading platform) among the points C. Side Ey1 connects two points C1 and C3 that are located in the negative X-axis direction (right edge of the loading platform) among the points C. Side Ey2 connects two points C2 and C4 that are located in the positive X-axis direction (left edge of the loading platform) among the points C.

[0027] The loading platform detection unit 12 detects a change in the detection target Ob in the loading platform from the loading platform image Gb. The detection target Ob is a baggage or a person. The loading platform detection unit 12 detects the detection target Ob in the loading platform 6b from the loading platform image Gb. The loading platform detection unit 12 detects the detection target Ob in the loading platform 6b based on image recognition technology. For example, the loading platform detection unit 12 performs image recognition processing such as template matching on the loading platform image Gb to recognize objects present in the image and their types. The loading platform detection unit 12 may detect a change in the detection target Ob by comparing the initial image Gi with the loading platform image Gb.

[0028] <Extrusion detection> The loading platform detection unit 12 detects whether the detection target Ob has fallen from the loading platform 6b or is about to fall from the loading platform 6b. The loading platform detection unit 12 detects a change in the detection target Ob, namely, whether the detection target Ob has protruded from the loading platform area E by an amount equal to or greater than a protrusion determination value. The detection target Ob has protruded from the loading platform area E when viewed on the XY plane. When the detection target Ob does not protrude from the loading platform area E, the detection target Ob exists within the loading platform area E when viewed on the XY plane. The detection of whether the detection target Ob has protruded from the loading platform area E by an amount equal to or greater than the protrusion determination value is performed using image recognition technology.

[0029] Figures 4, 5, 6 and 7 are schematic diagrams of the loading platform 6b as viewed from the front of the vehicle. Figure 4 is a diagram of the loading platform 6b immediately after luggage has been loaded onto it. In Figure 4, two detection targets Ob (detection targets Ob1 and Ob2) are arranged within the loading platform area E. These detection targets Ob do not extend beyond the loading platform 6b. The loading platform detection unit 12 saves the loading platform image Gb in the state shown in Figure 4 in memory as a loaded image. The loaded image is the loading platform image Gb immediately after luggage has been loaded onto the loading platform 6b. The state immediately after luggage has been loaded onto the loading platform 6b is referred to as "immediately after loading."

[0030] The bed detection unit 12 determines whether loading has occurred immediately after loading by comparing it with past bed images Gb stored in memory. For example, if no detection object Ob was detected in the past bed image Gb, but a new detection object Ob is detected in the latest bed image Gb, the bed detection unit 12 determines that loading has occurred immediately after loading. The bed detection unit 12 also determines that loading has occurred immediately after loading if a new detection object Ob is detected when the vehicle is restarted. The restart image Gr and the loaded image can be the same. The loaded image is used as a comparison image when detecting changes in the bed 6b, such as overhang detection and sway detection.

[0031] The loading platform detection unit 12 detects two detection targets Ob1 and Ob2 in FIG. 4 based on the loaded image. The loading platform detection unit 12 detects the edges (contours) of the detection targets Ob1 and Ob2 in the loaded image by edge extraction. The loading platform detection unit 12 detects whether the edges of the detection targets Ob1 and Ob2 protrude beyond the loading platform area E by more than the protrusion determination value. In the state of FIG. 4, the detection targets Ob1 and Ob2 do not protrude beyond the loading platform area E by more than the protrusion determination value, so no process is performed to notify the user.

[0032] FIG. 5 shows the state of the loading platform 6b after a predetermined time has elapsed since FIG. 4. FIG. 5 is a diagram showing the vehicle 6 while it is traveling. FIG. 5 shows an example in which the detection target Ob2 protrudes from the loading platform 6b. The detection target Ob2 is leaning against the edge of the loading platform 6b due to shaking while traveling, and is in a state in which it is likely to fall off. The tip of the detection target Ob2 protrudes outward in the X direction from the side Ey2 of the loading platform area E by a protrusion amount ΔT. The protrusion amount ΔT is the length by which the detection target Ob protrudes from the loading platform area E. The protrusion amount ΔT may also be the area by which the detection target Ob protrudes from the loading platform area E.

[0033] The loading platform detection unit 12 detects two detection targets Ob1 and Ob2 based on the latest acquired loading platform image Gb. The loading platform detection unit 12 detects whether the edges of the detection targets Ob1 and Ob2 protrude from the loading platform area E by an amount equal to or greater than the protrusion determination value. The loading platform detection unit 12 detects that the edge of the detection target Ob2 protrudes in the X direction by an amount ΔT. The loading platform detection unit 12 determines whether the amount ΔT of protrusion of the detection target Ob2 is greater than the protrusion determination value.

[0034] When the loading platform detection unit 12 detects that the protrusion amount ΔT of the detection object Ob2 is equal to or greater than the protrusion determination value, it outputs a signal to the notification processing unit 13 to notify the user. The protrusion determination value is a threshold value used to detect protrusion of the detection object Ob. The protrusion determination value is set in advance. The protrusion determination value may be set to any value.

[0035] FIG. 6 shows the state of the loading platform 6b after a predetermined time has elapsed since FIG. 5. FIG. 6 is a diagram showing the vehicle 6 while it is traveling. FIG. 6 shows an example in which the detection object Ob2 has fallen off the loading platform 6b. The detection object Ob2 is in a state in which the detection object Ob has completely fallen off the loading platform 6b due to shaking while traveling, etc. The detection object Ob2 is located outside the loading platform area E on the XY plane. The detection object Ob2 is located outside the imaging range of the imaging unit 5.

[0036] The bed detection unit 12 detects the detection object Ob1 based on the latest bed image Gb acquired. At this time, the detection object Ob2 is not detected because it is outside the imaging range of the imaging unit 5. The bed detection unit 12 compares the detection result in the latest bed image Gb with the detection result in a bed image Gb taken before the latest bed image Gb (hereinafter, a historical image). The historical image also includes a loaded image. The bed detection unit 12 detects that the detection object Ob2 has fallen based on the fact that the detection object Ob2 was detected in the historical image but was not detected in the latest bed image Gb. The bed detection unit 12 outputs a signal to the notification processing unit 13 to notify the user that the detection object Ob2 has fallen.

[0037] <Human detection> The loading platform detection unit 12 detects the presence of a person in the loading platform 6b as a change in the detection object Ob. FIG. 7 shows an example in which a person is present in the loading platform 6b. The loading platform detection unit 12 detects the detection object Ob3 based on the loading platform image Gb and determines the type of the detection object Ob3. Here, whether the detection object Ob is a person is determined by image recognition technology such as template matching. If the loading platform detection unit 12 determines that the detection object Ob3 is a person and that it is present in the loading platform 6b, it outputs a signal to the notification processing unit 13 to notify the user.

[0038] <Wobble detection> The loading platform detection unit 12 detects a change in the detection object Ob when the detection object Ob is swaying by more than the sway determination value. FIG. 8 shows an example in which the detection object Ob1 is swaying due to shaking while traveling. The solid line in FIG. 8 shows the detection object Ob1 at a certain time t1. The dashed line shows the detection object Ob1 at a time t2, a predetermined time after the time t1. In FIG. 8, the detection object Ob1 sways in the X direction by a sway amount Δτ between the time t1 and the time t2. The sway amount Δτ is the amplitude, or length, of the swing of the detection object Ob.

[0039] The platform detection unit 12 determines the sway of the detection object Ob based on the platform image Gb at time t1 and time t2. The platform detection unit 12 detects the detection object Ob1 based on the platform image Gb at time t1 and detects the edges of the detection object Ob1. The platform detection unit 12 detects the detection object Ob1 based on the platform image Gb at time t2 and detects the edges of the detection object Ob1. The platform detection unit 12 determines whether the detection object Ob1 detected based on the platform image Gb at time t1 and the detection object Ob1 detected based on the platform image Gb at time t2 are the same object. If the platform detection unit 12 determines that they are the same object, it performs a process to detect the amount of sway Δτ of the detection object Ob1.

[0040] The platform detection unit 12 compares the edge of the detection object Ob1 detected based on the platform image Gb at time t1 with the edge of the detection object Ob1 detected based on the platform image Gb at time t2, and detects the amount of sway Δτ of the detection object Ob1. If the amount of sway Δτ of the detection object Ob1 is equal to or greater than a sway determination value, the platform detection unit 12 outputs a signal to the notification processing unit 13 to notify the user. The sway determination value is a threshold value used to detect sway of the detection object Ob. The sway determination value is set in advance. The sway determination value may be set to any value.

[0041] <Detecting changes between stopping and restarting the vehicle> The bed detection unit 12 detects whether a change in the detection target Ob has occurred between the stopped image Gs (mainly an image immediately after stopping) and the restart image Gr as a change in the detection target Ob. The bed detection unit 12 detects the detection target Ob immediately after stopping based on the image immediately after stopping. The bed detection unit 12 detects the detection target Ob when starting based on the restart image Gr.

[0042] The cargo bed detection unit 12 detects whether a change has occurred in the detection targets Ob between the detection targets Ob detected based on the image immediately after the vehicle has stopped (hereinafter referred to as the detection targets when the vehicle is stopped) and the detection targets Ob detected based on the image Gr of the vehicle restarting (hereinafter referred to as the detection targets when the vehicle is restarting). A change in the detection targets Ob occurs, for example, when the number of detection targets when the vehicle is stopped and the number of detection targets when the vehicle is restarting has changed. If the number of detection targets when the vehicle is stopped decreases compared to the number of detection targets when the vehicle is stopped, it is possible that the detection targets Ob have fallen due to the influence of the wind or have been stolen.

[0043] Another example of a case where a change has occurred in the detection target Ob is when the detection target when the vehicle is stopped and the detection target when the vehicle is restarted are the same object and the position of the object has moved by more than a predetermined value. When the loading platform detection unit 12 detects that a change has occurred in the detection target Ob between the stopped vehicle image Gs and the restarting vehicle image Gr, it outputs a signal to the notification processing unit 13 to notify the user. <Detecting changes while parked>

[0044] The loading platform detection unit 12 detects changes in the detection target Ob while the vehicle is stopped. The loading platform detection unit 12 detects whether a change has occurred in the detection target Ob between multiple stopped vehicle images Gs as a change in the detection target Ob. The loading platform detection unit 12 detects whether a change has occurred in the detection target Ob based on multiple stopped vehicle images Gs. The loading platform detection unit 12 detects whether a change has occurred in the detection target Ob between the detection target Ob detected based on the latest acquired stopped vehicle image Gs and the detection target Ob detected based on a stopped vehicle image Gs acquired before the latest stopped vehicle image Gs. When the loading platform detection unit 12 detects that a change has occurred in the detection target Ob, it outputs a signal to the notification processing unit 13 to notify the user.

[0045] The periphery monitoring ECU 1 also performs processing to recognize the driving environment of the vehicle based on the detection results of sensors that monitor the periphery. For example, the periphery monitoring ECU 1 recognizes road signs and the presence or absence of obstacles from images acquired by a camera that captures the view ahead of the vehicle 6 through image recognition processing such as pattern matching. The periphery monitoring ECU 1 controls the display unit 3 to output a periphery monitoring image N (see FIGS. 13 and 14) related to the periphery monitoring system for autonomous driving.

[0046] When the loading platform detection unit 12 detects a change in the detection target Ob, the notification processing unit 13 performs processing to notify the user of the vehicle 6 of the change in the detection target Ob by at least one of sound output by the audio output unit 4 and display of a notification image Pi by the display unit 3. The notification processing unit 13 determines the notification mode. The notification processing unit 13 determines at least one of sound output and notification image Pi as the notification mode. The notification mode is determined depending on the state of the vehicle 6 or the content of the change in the detection target Ob, etc. The notification processing unit 13 determines the timing of the notification. The timing of the notification is determined depending on the state of the vehicle 6 or the content of the change in the detection target Ob, etc.

[0047] The notification processing unit 13 generates sound output data for causing the audio output unit 4 to output sound. The notification processing unit 13 generates image output data for causing the display unit 3 to display a notification image Pi. The sound output data includes data related to the sound to be output and information indicating the timing of the sound output. The image output data includes data on the notification image Pi and information indicating the display timing.

[0048] The sound output by the audio output unit 4 may be set to any sound that notifies the user of the vehicle 6 of a change in the loading platform 6b. The sound output by the audio output unit 4 may be a notification sound (warning sound) of a predetermined pattern or a voice message.

[0049] The notification image Pi displayed on the display unit 3 may be set arbitrarily as long as it is a display for informing the user of the vehicle 6 of a change in the loading platform 6b. As an example, the notification image Pi is an image generated mainly using the loading platform image Gb. The notification image Pi displays icons A1 to A5 on the loading platform image Gb. The icons A1 to A5 are icons indicating a change in the loading platform 6b. The notification image Pi displays a pop-up Po on the loading platform image Gb.

[0050] 9 is an example of a notification image Pi when protrusion is detected. In the notification image Pi, an icon A1 is superimposed on a loading platform image Gb. The icon A1 is an image that highlights the detection target Ob2 for which protrusion has been detected.

[0051] FIG. 10 is an example of a notification image Pi when the detection target Ob falls. In the notification image Pi, an icon A2 is superimposed on the loading platform image Gb. The icon A2 is an image that highlights the detection target Ob2 that has been detected to have fallen from the loading platform 6b. The icon A2 is generated by extracting the outline of the detection target Ob2 before it fell from the history image.

[0052] 11 is an example of a notification image Pi when a person is detected in the loading platform 6b. In the notification image Pi, an icon A3 is superimposed on the loading platform image Gb. The icon A3 is an arrow-shaped image that highlights the detected detection object Ob3 (person).

[0053] 12 shows an example of a notification image Pi when swaying is detected. In the notification image Pi, an icon A4 is superimposed on a loading platform image Gb. The icon A4 is displayed near the detection targets Ob1 and Ob2 for which swaying has been detected, and is an arrow-shaped image indicating the swaying of these targets.

[0054] 13 is an example of a notification image Pi when protrusion and falling are detected. A periphery monitoring image N related to a periphery monitoring system for autonomous driving is displayed on the display unit 3. In the notification image Pi, an icon A5 is superimposed on the periphery monitoring image N. The icon A5 is an image indicating that protrusion has been detected.

[0055] 14 is another example of a notification image Pi when overhang and a fall are detected. In the notification image Pi, a pop-up Po is superimposed on the surrounding monitoring image N. The pop-up Po is an image indicating that overhang has been detected. The pop-up Po includes a character string indicating that a package has fallen from the loading platform 6b.

[0056] <Flowchart for detecting overhang> The loading platform monitoring process shown in Figure 16 starts when the ignition switch of the vehicle 6 is turned on. The loading platform monitoring process is a process that monitors the loading platform 6b, and when a change in the detection object Ob is detected, notifies the user of the vehicle 6 of the change in the detection object Ob. The loading platform monitoring process includes detection of overhang, detection of people, detection of swaying, detection of changes between when the vehicle is stopped and when it is restarted, and detection of changes while the vehicle is stopped. The loading platform monitoring process is repeated while the ignition switch is on. The same applies to the other flowcharts in Figures 17 to 19.

[0057] In S21, the image acquisition unit 10 acquires a loading platform image Gb. In S22, the loading platform detection unit 12 performs overhang detection. The overhang detection process in S22 is mainly performed before the vehicle 6 starts or while the vehicle 6 is traveling, but may also be performed while the vehicle is stopped. The loading platform detection unit 12 detects a change in the detection target Ob based on the most recent loaded platform image Gb acquired, namely, overhang of the detection target Ob from the loading platform area E by an amount equal to or greater than the overhang determination value. In S23, the notification processing unit 13 performs control to notify the user of the change in the detection target Ob. The notification processing unit 13 determines the notification mode. As an example, here, the notification processing unit 13 performs control to both output sound and display a notification image Pi.

[0058] The notification processing unit 13 determines the timing of notification. The notification processing unit 13 determines the timing of sound output and image output. When the vehicle 6 is stopped before starting, the notification processing unit 13 controls to issue a notification when the shift lever is switched to a setting other than P. The notification processing unit 13 generates sound output data for notifying that there is a detection target Ob that has protruded beyond the loading platform area E. The notification processing unit 13 generates image output data for notifying that there is a detection target Ob that has protruded beyond the loading platform area E. The notification processing unit 13 generates an image as a notification image Pi in which an icon A1 that highlights the detection target Ob whose protrusion has been detected is superimposed on the loading platform image Gb as shown in FIG. 9.

[0059] The notification processing unit 13 transmits sound output data to the audio output unit 4. The notification processing unit 13 transmits image output data to the display unit 3. The display unit 3 displays a notification image Pi when the shift lever is switched to a setting other than P. The audio output unit 4 outputs sound when the shift lever is switched to a setting other than P.

[0060] The timing of sound output and display of the notification image Pi may be any other timing. The timing of sound output and image output may be immediately after overflow detection in S21. The notification to the user may be either sound output or display of the notification image Pi. The notification processing unit 13 may output sound and then display the notification image Pi. The notification processing unit 13 may output sound after displaying the notification image Pi. The notification processing unit 13 may output sound while the vehicle 6 is traveling, and then display the notification image Pi in response to a user operation.

[0061] <Human detection flowchart> 17 is a flowchart of human detection. In S31, the periphery monitoring ECU 1 determines whether the vehicle 6 is stopped. The periphery monitoring ECU 1 determines that the vehicle is stopped when the shift lever is set to P. The periphery monitoring ECU 1 may also determine that the vehicle is stopped when the shift lever is set to P and the parking brake is on.

[0062] In S32, the image acquisition unit 10 acquires a stopped vehicle image Gs. In S33, the cargo bed detection unit 12 performs human detection. The cargo bed detection unit 12 determines whether a human is present in the cargo bed 6b based on the most recently acquired stopped vehicle image Gs. If the answer is Yes in S32, the notification processing unit 13 determines the timing of notification so as to notify when the shift lever is switched to a position other than the P setting.

[0063] The notification processing unit 13 generates sound output data for notifying that a person has been detected in the loading platform 6b. The notification processing unit 13 generates image output data for notifying that a person has been detected in the loading platform 6b. The notification processing unit 13 generates an image as a notification image Pi in which an icon A3 emphasizing the detected person is superimposed on the loading platform image Gb as shown in FIG. 11 . The notification processing unit 13 transmits the sound output data to the audio output unit 4. The notification processing unit 13 transmits the image output data to the display unit 3.

[0064] In S34, the shift position sensor 20 detects that the shift lever has been changed to a position other than P. The state detection unit 2 transmits a signal to the audio output unit 4 and the display unit 3 notifying them that the shift lever has been changed to a position other than P. In S35, the display unit 3 displays a notification image Pi, and the audio output unit 4 outputs a sound.

[0065] The timing of the notification to the user may be any other timing. The timing of the sound output and image output may be immediately after human detection in S33. The notification to the user may be either a sound output or a display of a notification image Pi. The notification processing unit 13 may output a sound after displaying the notification image Pi. The notification processing unit 13 may first output a sound, and then display the notification image Pi in response to a user operation.

[0066] <Flowchart for detecting wobble> 18 is a flowchart of sway detection. In S41, the image acquisition unit 10 acquires a bed image Gb. In S42, the bed detection unit 12 performs sway detection. The sway detection process in S42 is mainly performed while the vehicle 6 is traveling, but may also be performed while the vehicle is stopped. The bed detection unit 12 detects sway of the detection object Ob that is equal to or greater than the sway determination value based on the most recent bed image Gb acquired.

[0067] In S43, the notification processing unit 13 performs control so as to notify the user of a change in the detection target Ob. The notification processing unit 13 determines the notification mode. As an example, it is assumed here that the vehicle 6 is traveling. The notification processing unit 13 first outputs sound, and then performs control so as to display a notification image Pi in response to a user operation.

[0068] The notification processing unit 13 determines the timing of notification. The notification processing unit 13 determines the timing of sound output. The notification processing unit 13 controls to output sound immediately. The notification processing unit 13 generates sound output data for notifying that swaying of the detection object Ob is equal to or greater than the sway judgment value. The notification processing unit 13 generates image output data for notifying that swaying of the detection object Ob is detected to be equal to or greater than the sway judgment value. The notification processing unit 13 generates an image as the notification image Pi, in which an icon A4 indicating the swaying of the detection object Ob whose swaying has been detected is superimposed on the loading platform image Gb as shown in FIG. 13.

[0069] The notification processing unit 13 transmits sound output data to the audio output unit 4. The notification processing unit 13 transmits image output data to the display unit 3. The audio output unit 4 outputs sound based on the sound output data. The display unit 3 displays a notification image Pi in response to an operation by the user.

[0070] The timing of outputting the sound and displaying the notification image Pi may be any other timing. The notification to the user may be either outputting the sound or displaying the notification image Pi. The notification processing unit 13 may output the sound after displaying the notification image Pi.

[0071] <Flowchart for detecting changes when the vehicle is stopped and when it starts moving again> In S51, the periphery monitoring ECU 1 determines whether the vehicle 6 is stopped. In S52, the image acquisition unit 10 acquires an image immediately after the vehicle has stopped. In S53, the bed detection unit 12 detects the detection object Ob based on the image immediately after the vehicle has stopped.

[0072] At S54, the shift position sensor 20 detects that the shift lever has been changed to a position other than P. The state detection unit 2 transmits a signal to the periphery monitoring ECU 1 notifying that the shift lever has been changed to a position other than P. At S55, the image acquisition unit 10 acquires the restart image Gr. At S56, the cargo bed detection unit 12 detects the detection object Ob based on the restart image Gr.

[0073] In S57, the loading platform detection unit 12 detects a change between when the vehicle is stopped and when it starts moving again. The loading platform detection unit 12 detects whether a change has occurred in the detection object Ob between the detection object when the vehicle is stopped and the detection object when it starts moving again. In S58, the notification processing unit 13 controls to notify the user of the change in the detection object Ob. The notification processing unit 13 determines the notification mode. As an example, here, the notification processing unit 13 controls to both output sound and display a notification image Pi. The notification processing unit 13 determines the timing of notification. The notification processing unit 13 determines the timing of outputting sound and displaying the notification image Pi. The notification processing unit 13 controls to immediately output sound and display the notification image Pi.

[0074] The notification processing unit 13 generates sound output data for notifying that a change has occurred in the detection object Ob between when the vehicle is stopped and when it starts moving again. The notification processing unit 13 generates image output data for notifying that a change has occurred in the detection object Ob between when the vehicle is stopped and when it starts moving again. The notification processing unit 13 generates an image as a notification image Pi, in which an icon emphasizing the detection object Ob in which a change has been detected is superimposed on the loading platform image Gb. The notification processing unit 13 transmits the sound output data to the audio output unit 4. The notification processing unit 13 transmits the image output data to the display unit 3.

[0075] The timing of the notification to the user may be any other timing. The notification to the user may be either a sound output or a display of a notification image Pi. The notification processing unit 13 may output a sound after displaying the notification image Pi. The notification processing unit 13 may first output a sound and then display the notification image Pi in response to a user operation.

[0076] In the above description of the flowchart, the bed detection unit 12 detects a change between when the vehicle is stopped and when it restarts based on the image immediately after stopping and the restarting image Gr, but this is not limited to this. The bed detection unit 12 may detect a change between when the vehicle is stopped and when it restarts based on the stopped image Gs other than the image immediately after stopping and the restarting image Gr.

[0077] <Flowchart for detecting changes while the vehicle is stopped> Here, the detection of a change while the vehicle is stopped by the periphery monitoring ECU 1 will be described (FIG. 20). In S61, the periphery monitoring ECU 1 determines whether the vehicle 6 is stopped. In S62, the image acquisition unit 10 acquires a stopped vehicle image Gs. In S63, the bed detection unit 12 detects the detection target Ob based on the most recently acquired stopped vehicle image Gs.

[0078] In S64, the bed detection unit 12 detects changes in the detection target Ob while the vehicle is stopped. The bed detection unit 12 detects whether a change has occurred in the detection target Ob based on a plurality of stopped vehicle images Gs. It detects whether a change has occurred in the detection target Ob between the detection target Ob detected based on the most recently acquired stopped vehicle image Gs and the detection target Ob detected based on a stopped vehicle image Gs acquired before the most recently acquired stopped vehicle image Gs. If there is no stopped vehicle image Gs acquired before the most recently acquired stopped vehicle image Gs, the flowchart ends.

[0079] If the answer is Yes in S64, the notification processing unit 13 performs control to notify the user of the change in the detection object Ob. The notification processing unit 13 determines the notification mode. Here, as an example, the notification processing unit 13 performs control to both output sound and display the notification image Pi. The notification processing unit 13 determines the timing of the notification. The notification processing unit 13 determines the timing of the sound output and the display of the notification image Pi. The notification processing unit 13 performs control to notify when the shift lever is switched to a position other than the P setting.

[0080] The notification processing unit 13 generates sound output data for notifying that a change has occurred in the detection object Ob while the vehicle is stopped. The notification processing unit 13 generates image output data for notifying that a change has occurred in the detection object Ob while the vehicle is stopped. The notification processing unit 13 generates an image as a notification image Pi, in which an icon emphasizing the detection object Ob in which a change has been detected is superimposed on the loading platform image Gb. The notification processing unit 13 transmits the sound output data to the audio output unit 4. The notification processing unit 13 transmits the image output data to the display unit 3.

[0081] In S65, the shift position sensor 20 detects that the shift lever has been changed to a position other than P. The state detection unit 2 transmits a signal notifying that the shift lever has been changed to a position other than P to the periphery monitoring ECU 1. In S66, the display unit 3 displays a notification image Pi, and the audio output unit 4 outputs sound.

[0082] The timing of the notification to the user may be any other timing. The timing of the sound output and image output may be immediately after the change is detected in S64. The notification to the user may be either a sound output or a display of a notification image Pi. The notification processing unit 13 may output a sound after displaying the notification image Pi. The notification processing unit 13 may first output a sound, and then display the notification image Pi in response to a user operation.

[0083] After the user is notified in S66, the sound output and the display of the notification image Pi may be stopped in response to an operation by the user.

[0084] <Summary of the embodiment> According to this embodiment, when a change in the detection target Ob in the loading platform 6b is detected from the loading platform image Gb obtained by capturing the loading platform 6b, the change in the detection target Ob is notified to the user by at least one of sound output and display of a notification image Pi. In this way, by notifying the user using the change in the detection target Ob as a trigger, the user can easily recognize the change in the detection target Ob in the loading platform 6b.

[0085] According to this embodiment, the user is notified that the detection target Ob has protruded beyond the loading platform area E by more than the protrusion determination value. This allows the user to know that the detection target Ob is about to fall outside the loading platform 6b or has actually fallen outside.

[0086] The area setting unit 11 sets the loading area E based on the operation information, thereby allowing the loading area E to be set according to the user's preference.

[0087] According to this embodiment, the user is notified that there is a person in the loading platform 6b, which allows the user to know that there is a person in the loading platform 6b, thereby making it possible to prevent an accident.

[0088] According to this embodiment, the user is notified that the detection target Ob is swaying more than the sway determination value, thereby letting the user know that the detection target Ob is not fixed firmly enough.

[0089] According to this embodiment, the user is notified that there has been a change in the detection object Ob between when the vehicle stopped and when it started moving again, thereby enabling the user to know that there has been a change in the detection object Ob between when the vehicle stopped and when it started moving again.

[0090] According to this embodiment, the user is notified that a change has occurred in the detection object Ob while the vehicle is stopped, thereby allowing the user to know that a change has occurred in the detection object Ob while the vehicle is stopped.

[0091] <Modification> Although the configuration in which the loading area E is set by the user has been shown, this is not necessarily limited thereto. The loading area E may also be set in advance at the time of shipping from the factory.

[0092] Although the configuration has been shown in which the area setting unit 11 sets the loading platform area E based on the four corner points C input by the user, this is not necessarily limited to this. The area setting unit 11 may also recognize the four corners of the loading platform 6b from the loading platform image Gb and automatically set the loading platform area E.

[0093] Although the periphery monitoring ECU 1 has been configured to notify the user through the notification processing unit 13 when it determines that a person is present in the loading platform 6b, this is not necessarily limited to this. The periphery monitoring ECU 1 may also be configured to notify the user when it determines that a person is present around the loading platform 6b.

[0094] Although the periphery monitoring ECU 1 has been shown to have a configuration including the image acquisition unit 10, the area setting unit 11, the bed detection unit 12, and the notification processing unit 13, this is not necessarily limited to this. The imaging unit 5 may also have some or all of the area setting unit 11, the bed detection unit 12, and the notification processing unit 13.

[0095] (Disclosure of technical ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be written in a multiple dependent form, with the subsequent clause referring to the preceding clause as an alternative. Furthermore, some clauses may be written in a multiple dependent form, referring to another multiple dependent clause. These multiple dependent clauses define multiple technical ideas. (Technical thought 1) A control device for use in a vehicle (6), an image acquisition unit (10) that acquires a bed image (Gb) captured by an imaging device (5) that captures an image of the bed (6b) of the vehicle; a loading platform detection unit (12) that detects a change in a detection object (Ob) within the loading platform from the loading platform image; and a notification processing unit (13) that, when the loading platform detection unit detects a change in the detection object, notifies the user of the vehicle of the change in the detection object by at least one of outputting sound from an audio output unit (4) and displaying a notification image from an image output unit (3). (Technical thought 2) a region setting unit (11) for setting a loading platform region (E) that is a region within the loading platform; The loading platform detection unit detects, as the change in the detection object, a protrusion of the detection object from the loading platform area by a protrusion determination value or more, The control device according to Technical Idea 1, wherein the notification processing unit notifies the user when the loading platform detection unit detects a protrusion equal to or greater than the protrusion determination value. (Technical Thought 3) The control device according to Technical Idea 2, wherein the area setting unit sets the loading platform area based on operation information of an operation by the user. (Technical Thought 4) the loading platform detection unit detects the presence of a person in the loading platform as the change in the detection target, The control device according to any one of technical ideas 1 to 3, wherein the notification processing unit notifies the user when the loading platform detection unit detects that a person is present in the loading platform. (Technical Thought 5) The loading platform detection unit detects a sway of the detection object that is equal to or greater than a sway determination value as a change in the detection object, The control device according to any one of Technical Ideas 1 to 4, wherein the notification processing unit notifies the user when the loading platform detection unit detects sway equal to or greater than a sway determination value. (Technical Thought 6) The image acquisition unit Acquire a stopped vehicle image (Gs) which is an image of the loading platform when the vehicle is stopped; Acquire a restart image (Gr) which is an image of the loading platform when the vehicle restarts; the loading platform detection unit detects whether a change has occurred in the detection target between the stopped image and the restarted image, The notification processing unit notifies the user when the loading platform detection unit detects that a change has occurred in the detection object between the stopped image and the restart image. (Technical Thought 7) the image acquisition unit acquires a plurality of stopped vehicle images (Gs) which are images of the loading platform when the vehicle is stopped; The control device described in any one of technical ideas 1 to 6, wherein the notification processing unit notifies the user when the loading platform detection unit detects that a change has occurred in the detection object between multiple stopped vehicle images. [Explanation of symbols]

[0096] 1 control device, 10 image acquisition unit, 11 area setting unit, 12 cargo bed detection unit, 13 notification processing unit, 14 processor, 3 display unit (image output unit), 4 audio output unit, 5 imaging unit (imaging device), 6 vehicle, 6b cargo bed, E cargo bed area, Gb cargo bed image, Gr restart image, Gs stopped image, Ob detection target, Pi notification image.

Claims

1. A control device for use in a vehicle (6), comprising: an image acquisition unit (10) that acquires a bed image (Gb) captured by an imaging device (5) that captures an image of the bed (6b) of the vehicle; a loading platform detection unit (12) that detects a change in a detection object (Ob) within the loading platform from the loading platform image; and a notification processing unit (13) that, when the loading platform detection unit detects a change in the detection object, notifies the user of the vehicle of the change in the detection object by at least one of outputting sound from an audio output unit (4) and displaying a notification image (Pi) from an image output unit (3).

2. a region setting unit (11) for setting a loading platform region (E) that is a region within the loading platform; The loading platform detection unit detects, as the change in the detection object, a protrusion of the detection object from the loading platform area by a protrusion determination value or more, The control device according to claim 1 , wherein the notification processing unit notifies the user when the loading platform detection unit detects a protrusion equal to or greater than the protrusion determination value.

3. The control device according to claim 2 , wherein the area setting unit sets the loading area based on operation information of an operation by the user.

4. the loading platform detection unit detects the presence of a person in the loading platform as the change in the detection target, The control device according to claim 1 , wherein the notification processing unit notifies the user when the loading platform detection unit detects that a person is present in the loading platform.

5. The loading platform detection unit detects a sway of the detection object that is equal to or greater than a sway determination value as a change in the detection object, The control device according to claim 1 , wherein the notification processing unit notifies the user when the loading platform detection unit detects a sway that is equal to or greater than a sway determination value.

6. The image acquisition unit Acquire a stopped vehicle image (Gs) which is an image of the loading platform when the vehicle is stopped; A restart image (Gr) is acquired, which is an image of the loading platform when the vehicle restarts; the loading platform detection unit detects whether a change has occurred in the detection target between the stopped image and the restarted image, The control device according to claim 1 , wherein the notification processing unit notifies the user when the loading platform detection unit detects that a change has occurred in the detection target between the stopped image and the restart image.

7. The image acquisition unit acquires a plurality of stopped vehicle images (Gs) which are images of the loading platform when the vehicle is stopped, The control device according to claim 1 , wherein the notification processing unit notifies the user when the loading platform detection unit detects that a change has occurred in the detection target among the plurality of images of the stopped vehicle.

8. A bed monitoring system for use in a vehicle (6), comprising: The vehicle is equipped with an imaging device (5) for photographing a loading platform (6b) of the vehicle and a control device (1), The control device an image acquisition unit (10) that acquires a loading platform image (Gb) captured by the imaging device; a loading platform detection unit (12) that detects a change in a detection object (Ob) within the loading platform from the loading platform image; A loading platform monitoring system having a notification processing unit (13) that, when the loading platform detection unit detects a change in the detection object, notifies the user of the vehicle of the change in the detection object by at least one of outputting sound from an audio output unit (4) and displaying a notification image (Pi) from an image output unit (3).

9. A notification method for use in a vehicle (6), comprising: A loading platform image (Gb) is acquired by an imaging device (5) that photographs the loading platform (6b) of the vehicle (S11); A change in the detection object (Ob) in the loading platform is detected from the loading platform image (S12); The notification method includes, in the processing executed by at least one processor (14), a step of notifying the user of the vehicle of the change in the detection target by at least one of outputting sound from an audio output unit (4) and displaying a notification image (Pi) from an image output unit (3) when a change in the detection target is detected (S13).

Images