Remote monitoring device and method

Abstract

The invention provides a remote monitoring device and a remote monitoring method. An image of a vehicle-mounted camera is remotely monitored from an external terminal. The remote monitoring device comprises a control part which enables the remote monitoring device to alternatively operate in a first mode and a second mode; and a communication device that communicates with the external terminal. In the first mode, the vehicle-mounted camera is connected to a display of a vehicle, and an image of the vehicle-mounted camera is displayed on the display. And a second mode in which the in-vehicle camera is not connected to the display but is connected to the communication device, and an image of the in-vehicle camera is displayed on the display of the external terminal through the communication device.

Description

Technical Field

[0001] The invention relates to a remote monitoring device and method. Background Art

[0002] In recent years, research and development that contributes to energy efficiency has been underway in order to ensure access to affordable, reliable, sustainable, and advanced energy for more people.

[0003] There is a system that can monitor the image of the vehicle-mounted camera from an external terminal outside the vehicle. Because it is hoped that the image of the vehicle-mounted camera can be monitored through the display of the vehicle when the ignition switch is turned on (the vehicle is in use).

[0004] Patent Documents 1 and 2 disclose monitoring images from an in-vehicle camera from outside a vehicle, but neither patent document discloses connecting to a smartphone via a communication device while bypassing an in-vehicle display.

[0005] Prior Art Literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-036067

[0008] Patent document 2: Japanese Patent Application No. 2018-101865

[0009] However, in the present technology, a problem remains as to how to easily switch the remote monitoring device between the in-vehicle display and the external terminal display. Summary of the invention

[0010] In order to solve the above-mentioned problems, the present invention aims to achieve easy switching between the display on the vehicle monitor and the display on the external terminal, thereby contributing to energy efficiency.

[0011] According to the above description, according to an embodiment of the present invention, a remote monitoring device is provided for remotely monitoring the image of a vehicle-mounted camera from an external terminal. The remote monitoring device includes: a control unit, which enables the remote monitoring device to select one of a first mode and a second mode; and a communication device, which communicates with the external terminal. In the first mode, the vehicle-mounted camera is connected to a display of the vehicle, so that the image of the vehicle-mounted camera is displayed on the display. In the second mode, the vehicle-mounted camera is not connected to the display but is connected to the communication device, and the image of the vehicle-mounted camera is displayed on the display of the external terminal through the communication device.

[0012] According to an embodiment of the present invention, the remote monitoring device further includes: a switching unit for switching between the first mode and the second mode.

[0013] According to an embodiment of the present invention, in the remote monitoring device, the switching unit automatically switches to the first mode or the second mode based on an on / off state of an ignition switch of the vehicle.

[0014] According to an embodiment of the present invention, in the remote monitoring device, when the ignition switch is on, the switching unit switches to the first mode, and when the ignition switch is off, the switching unit switches to the second mode.

[0015] According to an embodiment of the present invention, in the remote monitoring device, when the ignition switch is in an on state and a remote monitoring request is received from the external terminal, the switching unit switches to the second mode.

[0016] According to an embodiment of the present invention, in the remote monitoring device, when a remote monitoring request is received from the external terminal, switching to the second mode is prohibited while the image captured by the vehicle-mounted camera is displayed on the display of the vehicle.

[0017] According to an embodiment of the present invention, in the remote monitoring device, in the second mode, when a request is received to display an image captured by the vehicle-mounted camera on the display of the vehicle, the remote monitoring device switches to the first mode.

[0018] According to another embodiment of the present invention, a remote monitoring method is provided, which remotely monitors the image of a vehicle-mounted camera from an external terminal. The remote monitoring method includes: determining the on / off state of an ignition switch of a vehicle; and switching the remote monitoring to a first mode or a second mode based on the on / off state of the ignition switch. The first mode is to display the image of the vehicle-mounted camera on a display of the vehicle. The second mode is to display the image of the vehicle-mounted camera on a display of the external terminal instead of displaying it on the display of the vehicle.

[0019] According to an embodiment of the present invention, in the remote monitoring method, when the ignition switch is in the on state, the mode is switched to the first mode; and when the ignition switch is in the off state, the mode is switched to the second mode.

[0020] According to an embodiment of the present invention, in the remote monitoring method, when the ignition switch is in an on state and a remote monitoring request is received from the external terminal, the switching unit switches to the second mode.

[0021] According to an embodiment of the present invention, in the remote monitoring method, when a remote monitoring request is received from the external terminal, switching to the second mode is prohibited while the image captured by the vehicle-mounted camera is displayed on the display of the vehicle.

[0022] According to an embodiment of the present invention, in the remote monitoring method, in the second mode, when a request is received to display an image captured by the vehicle-mounted camera on the display of the vehicle, the method switches to the first mode.

[0023] According to the embodiment of the present invention, in the remote monitoring device, the vehicle-mounted display is used for monitoring when the ignition switch is turned on (vehicle in use), and the display of the external terminal outside the vehicle is used for monitoring when the ignition switch is turned off (vehicle not in use), and the two can be easily switched. Moreover, the remote monitoring device can automatically switch between the first mode and the second mode according to the on / off state of the vehicle's ignition switch through the switching unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the architecture of an image playback system to which the remote monitoring device and method according to the embodiments of the present invention can be applied.

[0025] Figure 2 A functional block diagram of a remote monitoring device according to an embodiment of the present invention is illustrated.

[0026] Figure 3A and Figure 3B This is a display example of a remote monitoring device.

[0027] Figure 4 This is a system architecture that can be adopted by the remote monitoring device of the embodiment of the present invention. DETAILED DESCRIPTION

[0028] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0029] Figure 1 It is a schematic diagram of the architecture of the image playback system of the remote monitoring device of the embodiment of the present invention. Figure 2 The functional block diagram of the remote monitoring device according to the embodiment of the present invention is shown as follows: The remote monitoring device 100 is mainly composed of a mobile object V and an external terminal T. The external terminal T can be used to monitor the surrounding conditions of the mobile object V remotely.

[0030] like Figure 1As shown, the remote monitoring device 100 can transmit multiple images obtained by using multiple cameras mounted on the mobile body V to the external terminal T of the user U located outside the mobile body V, and play or display them on the display of the external terminal T. The external terminal T can be a portable terminal device such as a smart phone or a tablet. The remote monitoring device 100 can communicate with the vehicle V through the communication network 10. The remote monitoring device 100 can also include a relay server 20 that is communicably connected to the vehicle via the communication network 10.

[0031] As an example, multiple images can be generated based on images from multiple cameras (i.e., onboard cameras), which are four images in different directions around the moving object V. The moving object V can be a vehicle, but is not limited to this. For example, it can be a moving object on the surface of water or underwater such as a ship, or a moving object in the sky or space. The following description takes the vehicle V as an example of the moving object V.

[0032] like Figure 2 As shown, the vehicle V side of the remote monitoring device 100 includes at least but not limited to a control unit 110, a camera control unit 120, an image processing unit 130, and a display control unit 140. The vehicle V may also include a camera 122 installed outside the vehicle, a display (such as a liquid crystal display LCD) installed inside the vehicle V, and an ignition switch 160 for starting or shutting down the vehicle V. The vehicle V may also include a communication device 150, which can communicate with the external terminal T of the user U.

[0033] In addition to controlling the remote monitoring of the present embodiment, the control unit 110 can also be implemented as a part of the entire vehicle control device by a processor, such as the Electronic Control Unit (ECU) of the vehicle V, or the core electronic control unit (core ECU). The ECU unit can be used to control various sensors of the vehicle. Various system controls in the vehicle V are controlled by the ECU unit. The control unit 110 can be used to control the above-mentioned parts. In addition, when the control unit 110 is used as an ECU unit, it can also be used to control the vehicle, such as controlling the driving support unit to control the driving support function. Thus, the control unit 110 can control various behavior modes of the vehicle, such as acceleration, deceleration, parking, avoidance, etc., and can also detect the vehicle state or the state of the surrounding area of ​​the vehicle by controlling various sensors and detectors. For those skilled in the art, various control methods of the control unit 110 can be designed according to actual needs.

[0034] According to an embodiment of the present invention, the remote monitoring device 100 is used to monitor images captured by a camera from an external terminal T. The control unit 110 is configured to allow the remote monitoring device 100 to operate in a first mode or a second mode. Figure 2 As shown, in the first mode, the camera 122 is connected to the display 142 (i.e., the transmission path of the signal) of the vehicle V, so that the image captured by the camera 122 is displayed on the display 142. Also, in the second mode, the camera 122 is connected to the communication device 150, and through the communication device 150 (i.e., the transmission path of the signal), the image captured by the camera 122 is displayed on the display of the external terminal T. Thus, in the second mode, the user U can remotely monitor the vehicle V through the external terminal T.

[0035] Also, in the first mode, the camera 122 (eg Figure 1 The images captured by the cameras 122a to 122d are transmitted to the display control unit 140 for processing and then displayed on the display 142. In this way, the user (driver) U can monitor the surroundings of the vehicle V from the display 142 on the center console of the vehicle V.

[0036] In the second mode, as Figure 1 and Figure 2 As shown, in this embodiment, the plurality of cameras 122 are, for example, four, namely, a left front camera 122a, a right front camera 122b, a left rear camera 122c, and a right rear camera 122d, which are respectively arranged at the front left side, the front right side, the rear left side, and the rear right side of the vehicle V. The camera control unit 120 can receive and process the images captured by each camera 122a to 122d. In this embodiment, the images generated by the plurality of cameras 122 are processed by the camera control unit 120 (for example, by the viewing image generating unit (not shown)) to generate a front image VF, a rear image VB, a left side image VL, and a right side image VR of the vehicle V that can be viewed by an external terminal (such as a smart phone) T, which can be referred to as specific direction images. These specific direction images VF, VB, VL, and VR can be generated by processing the images generated by the plurality of cameras 122 using any executable image processing method.

[0037] The image processing unit 130 may be composed of a processor and a memory, and may transmit the image information processed by the camera control unit 120 to the terminal of the user U through the relay server 20. At this time, the image processing unit 130 may also perform various processes related to image processing such as image compression.

[0038] Figure 3A and Figure 3BThis is an example of a display of a remote monitoring device. As mentioned above, the remote monitoring device 100 is implemented by an external terminal 8 such as a smart phone, and the monitoring image is displayed in a 4-split (field-shaped) manner as an example of a monitoring display screen. Figure 3A As shown, as an example, the remote monitoring image screen can be displayed on the touch screen 32 of the external terminal 8. The remote monitoring image screen includes a thumbnail display 50, a thumbnail display key 52 and an end key 53. The remote monitoring image screen can also include a date and time display 54 and a location display 55. The thumbnail display 50 is a display of images in four directions as thumbnails of still images.

[0039] The remote monitoring image screen may also include a selection operation unit 51, which is centered on a dotted body 60 that simulates the body of the vehicle V and displays an image selection key 61a for specifying a front image VF, an image selection key 61b for specifying a right image VR, an image selection key 61c for specifying a rear image VB, and an image selection key 61d for specifying a left image VL, wherein the image selection keys 61a to 61d are collectively referred to as image selection keys 61.

[0040] The thumbnail display key 52 is a key for displaying the thumbnail display 50 on the touch screen. Figure 3B The thumbnail display 50 is displayed on the screen shown without the thumbnail display 50. Figure 3A In the example, since the thumbnail display 50 is already displayed, the thumbnail display key 52 is in an invalid state. The end key 53 is used to end the image playback application (application, app).

[0041] exist Figure 3A On the screen shown, the user U determines an image in one direction to be enlarged and displayed from the thumbnail display 50, and issues a selection instruction by touching the image selection key 61 (e.g., one of the image selection keys 61a to 61d) corresponding to the image in the determined direction. In this way, the image in the specific direction captured by the camera of the vehicle V can be displayed on the display screen of the external terminal 8 of the user U.

[0042] like Figure 3B In the example shown in FIG. 1 , when the user U presses the image selection key 61a showing the image in front of the vehicle V among the plurality of image selection keys 61, the touched image selection key 61a showing the image in front of the vehicle V turns gray. Figure 3A The specific direction image display 56 (in this example, the front image of the vehicle 2) of the direction is displayed at the position displayed by the thumbnail display 50 of the image selection key 61. Therefore, the specific direction image selected by the image selection key 61 is enlarged and displayed. Figure 3BIn the screen shown, the thumbnail display key 52 is not grayed out but is displayed normally. The user U can touch the thumbnail display key 52 to restore the screen of the touch screen 32 to Figure 3A The screen shown.

[0043] According to an embodiment of the present invention, the display control unit 140 may include a switching unit 144 for switching between the first mode and the second mode. For example, the switching between the first mode and the second mode may be performed by switching the image signal transmission path. The switching unit 144 may, for example, use a physical switch circuit to switch the signal path, or any other suitable method, which is not particularly limited by the present invention.

[0044] The switching method of the switching unit can determine whether the remote monitoring device 100 is to execute the first mode or the second mode according to the state of the vehicle V. In other words, according to the on / off state of the ignition switch 160 of the vehicle V, the control unit 110 can send a control signal to the switching unit 144 to switch the signal transmission path. Thus, the switching unit 144 can automatically switch the remote monitoring device 100 to the first mode or the second mode based on the on / off state of the ignition switch 160 of the vehicle V.

[0045] When the ignition switch 160 is in the on state, that is, the user (driver) U of the vehicle V is already in the driver's seat and is ready to drive the vehicle V. At this time, it is sufficient to monitor the surroundings of the vehicle using the display 142 on the center console of the vehicle V. Therefore, the switching unit 144 can switch the remote monitoring device 100 to the first mode based on the ignition switch 160 being in the on state. That is, the remote monitoring device 100 uses the display 142 on the vehicle V for monitoring.

[0046] Furthermore, when the ignition switch 160 is turned off, that is, the user (driver) U of the vehicle V may have left the vehicle, is not in the driver's seat, or has moved away from the vehicle V. At this time, the user U preferably uses the external terminal T held by the user to monitor the surroundings of the vehicle V. Therefore, the switching unit 144 can switch the remote monitoring device 100 to the second mode based on the ignition switch 160 being turned off. That is, the remote monitoring device 100 uses the display of the external terminal T for monitoring, that is, Figure 3A and Figure 3B The method described.

[0047] In addition, when the ignition switch 160 is in the started state and a remote monitoring request is received from the external terminal T, the switching unit 144 can switch the remote monitoring device 100 to the second mode for execution. That is, although the vehicle V is in the started state, the user may not be in the vehicle, but near the outside of the vehicle V. At this time, the user U may still have the need to perform remote monitoring from the outside of the vehicle V. Therefore, even when the vehicle V is started, the user can still send a remote monitoring request. Therefore, through the switching unit 144, the remote monitoring device 100 can operate in the second mode.

[0048] Furthermore, when a remote monitoring request is received from the external terminal T, if the image captured by the camera 122 has been displayed on the display 142 of the vehicle V, during the display of the display 142, the control unit 110 can disable the switching unit 144 to prohibit the remote monitoring device 100 from switching to the second mode. This mode is the so-called vehicle display priority mode. That is, the remote monitoring device 100 of the present invention takes the display of the display 142 as a priority option.

[0049] Furthermore, if the remote monitoring device 100 is currently operating in the second mode, the remote monitoring device 100 is in remote monitoring with the external terminal T. At this time, if the remote monitoring device 100 receives a request to display the image captured by the camera 122 on the display 142 of the vehicle V, the control unit 110 will enable the switching unit 144 to switch the operating second mode to the first mode. In other words, the remote monitoring device 100 of the present invention takes the display of the display 142 as a priority option.

[0050] Figure 4 This is a system architecture that can be used by the remote monitoring device of the embodiment of the present invention. Figure 4As shown, the system architecture may include multiple ECU units, such as a multi-view camera (MVC) ECU unit 200, a center information display (CID) 202, a cockpit domain controller (CDC) 204, a core ECU 206, a telematic control unit (TCU) 208, and an area ECU center 209. Here, each control unit and ECU unit may communicate via a so-called controller area network (CAN or CAN bus). The controller area network is a multifunctional automotive bus standard designed to allow single chips and instruments on the network to communicate with each other without the need for a host.

[0051] The MVC ECU unit 200 can be used to control the above-mentioned multiple cameras 122, and the MVC ECU unit 200 can perform the function of the above-mentioned image viewing generation unit. That is, the MVC ECU unit 200 can be used as a viewing image generation unit to generate a front image VF, a rear image VB, a left side image VL, and a right side image VR of the vehicle V that can be viewed by an external terminal (such as a smart phone) T. According to one embodiment of the present invention, the MVC ECU unit 200 can be used to implement the above-mentioned camera control unit 120.

[0052] The central information display (CID) 202 may include a liquid crystal display (LCD) 202a. The liquid crystal display (LCD) 202a may be used to display images captured by the plurality of cameras 122, display various vehicle information, display navigation maps, display various applications executable in the vehicle V, etc. According to one embodiment of the present invention, the central information display (CID) 202 may be used to implement the display control unit 142 described above.

[0053] like Figure 4As shown, the switching unit 144 can switch between signal transmission paths I and II. The switching unit 144 can be implemented by a general switch. Here, the signal transmission path I is to process the image signal captured by the camera 1122 first by the MVC ECU unit 200, and then by the CID 202 (such as using a video processor not shown), and then display it on the display (such as LCD) 142. The signal transmission path II is to transmit the image signal captured by the camera 1122 directly to the core ECU unit 206 via the CDC 204 without processing by the CID 202. After that, it is transmitted to the external terminal T via the TCU unit 208.

[0054] The driving domain controller (CDC) 204 can integrate the functions of multiple different operating systems and security levels into one platform, and meet the needs of users of the vehicle V with advantages such as high integration, high performance, and high scalability. The driving domain controller (CDC) 204 can also serve as the above-mentioned image processing unit 130 (see Figure 2 ), which can perform functions such as image compression, image transmission and reception, etc.

[0055] The core ECU 206 can be used as the control unit 110 described above. In addition to executing functions such as the switching unit 112 of remote monitoring, it can also perform various control functions of the straight-line vehicle V. The core ECU 206 can communicate with the relay server 210 through a telematics control unit (TCU) 208. According to one embodiment of the present invention, the telematics control unit (TCU) 208 can be used to implement the above-mentioned communication device 150.

[0056] The telematics control unit (TCU) 208 and the relay server 210 may communicate using a 4G or 5G protocol. Afterwards, the relay server 210 communicates with the external terminal T of the user U. Thus, the external terminal T can remotely monitor the vehicle V.

[0057] The regional ECU center 209 may have a security alarm function, for example.

[0058] The above system architecture is merely an executable example and is not intended to limit the implementation of the present invention. The system architecture may be appropriately changed and modified based on the concepts of the present invention.

[0059] According to an embodiment of the present invention, monitoring is performed using the on-board display when the ignition switch is turned on (vehicle in use), and monitoring is performed using the display of an external terminal outside the vehicle when the ignition switch is turned off (vehicle not in use), and the two can be easily switched.

[0060] According to an embodiment of the present invention, the remote monitoring device and method can identify the country where the vehicle is located. The remote monitoring device and method can further determine whether the country can provide remote monitoring services. If this service cannot be provided, the user can be automatically notified or remote monitoring can be automatically stopped. Therefore, through the present invention, a more convenient operation can be provided for whether to stop remote monitoring to reduce the inconvenience of the user.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote monitoring device for remotely monitoring the image of a vehicle-mounted camera from an external terminal, characterized in that: include: The control unit causes the remote monitoring device to operate in either the first mode or the second mode. a communication device for communicating with the external terminal, wherein in the first mode, the vehicle-mounted camera is connected to a display of the vehicle so that the image of the vehicle-mounted camera is displayed on the display, In the second mode, the vehicle-mounted camera is not connected to the display but is connected to the communication device, and the image of the vehicle-mounted camera is displayed on the display of the external terminal through the communication device.

2. The remote monitoring device according to claim 1, characterized in that: Also includes: The switching unit is used for switching between the first mode and the second mode.

3. The remote monitoring device according to claim 2, characterized in that: The switching unit automatically switches to the first mode or the second mode based on an on / off state of an ignition switch of the vehicle.

4. The remote monitoring device according to claim 3, characterized in that: When the ignition switch is in the on state, the switching unit switches to the first mode. When the ignition switch is turned off, the switching unit switches to the second mode.

5. The remote monitoring device according to claim 3 or 4, characterized in that: When the ignition switch is in the on state and a request for remote monitoring is received from the external terminal, the switching unit switches to the second mode.

6. The remote monitoring device according to claim 5, characterized in that: When a remote monitoring request is received from the external terminal, switching to the second mode is prohibited while the image captured by the vehicle-mounted camera is displayed on the display of the vehicle.

7. The remote monitoring device according to claim 5, characterized in that: In the second mode, when a request is received to display an image captured by the vehicle-mounted camera on the display of the vehicle, the mode is switched to the first mode.

8. The remote monitoring device according to claim 6, characterized in that: In the second mode, when a request is received to display an image captured by the vehicle-mounted camera on the display of the vehicle, the mode is switched to the first mode.

9. A remote monitoring method for remotely monitoring the image of a vehicle-mounted camera from an external terminal, characterized in that: include: Determine whether the vehicle's ignition switch is on / off; as well as Based on the on / off state of the ignition switch, the remote monitoring is switched to the first mode or the second mode, The first mode is to display the image of the vehicle-mounted camera on the display of the vehicle, The second mode is to display the image of the vehicle-mounted camera on the display of the external terminal instead of displaying the image on the display of the vehicle.

10. The remote monitoring method according to claim 9, characterized in that: When the ignition switch is in the on state, switching to the first mode; and When the ignition switch is in the off state, the mode is switched to the second mode.

11. The remote monitoring method according to claim 9 or 10, characterized in that: When the ignition switch is in the on state and a request for remote monitoring is received from the external terminal, the switching unit switches to the second mode.

12. The remote monitoring method according to claim 11, characterized in that: When a remote monitoring request is received from the external terminal, switching to the second mode is prohibited while the image captured by the vehicle-mounted camera is displayed on the display of the vehicle.

13. The remote monitoring method according to claim 11, characterized in that: In the second mode, when a request is received to display an image captured by the vehicle-mounted camera on the display of the vehicle, the mode is switched to the first mode.

Images