Method for remotely monitoring videos inside and outside vehicle, cloud server, vehicle and system

Abstract

The invention discloses a method for remotely monitoring videos inside and outside a vehicle, a cloud server, the vehicle and a system. The method is applied to the cloud server. Wherein the cloud server comprises an audio-video cloud server and a vehicle control cloud server which are mutually independent; the method comprises the following steps: when a mobile terminal has a video viewing demand, transmitting a function starting command actively issued by the mobile terminal to a vehicle bound with the mobile terminal through a control link so as to wake up the vehicle; wherein the control link is a control parameter transmission link at least composed of a vehicle control cloud server and a vehicle; transmitting a video control parameter actively issued by the mobile terminal to the vehicle through the video transmission link, so that the vehicle shoots a target video clip according to the video control parameter; wherein the video transmission link is a multimedia data transmission link at least composed of an audio-video cloud server and a vehicle; and transmitting the target video clip to the audio-video cloud server through the video transmission link, and transmitting the target video clip to the mobile terminal for viewing by the audio-video cloud server.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle remote monitoring, and in particular to a method, a cloud server, a vehicle, and a system for remotely monitoring videos inside and outside a vehicle. Background Art

[0002] With the continuous development of automobile functions, in-vehicle products such as driving recorders, GPS in-vehicle terminals, and in-vehicle entertainment systems are constantly evolving. Users are no longer limited to basic functions such as vehicle driving information collection, positioning data collection, and picture capture. Instead, they expect to record video images in real time, remotely obtain video images inside and outside the vehicle, and realize visual supervision of the vehicle's driving and parking status.

[0003] Current remote monitoring technologies typically centralize vehicle control commands and video stream data on the same server for interaction with the vehicle. Because vehicle control commands require low-latency, highly reliable transmission to ensure driving safety, and video streams are large and resource-intensive, uploading both to the same server can easily lead to delayed response times for vehicle control commands, lags in video transmission, or even interruptions due to resource competition. Furthermore, server attacks can potentially leak both control data related to vehicle control and video data containing private information, posing a serious threat to the privacy and security of the vehicle and its owner. Summary of the Invention

[0004] In order to solve or partially solve the technical problems of resource crowding and low security caused by the centralized interaction of vehicle body control instructions and video stream data on the same server and vehicle end in the existing remote monitoring technology, the present invention provides a method, cloud server, vehicle and system for remotely monitoring videos inside and outside the vehicle. Different links are used for transmission of control instructions and video data, which can meet the real-time and low-latency communication requirements of vehicle body control, and the high bandwidth and high network load requirements of video transmission, thereby significantly improving the real-time and security of monitoring. It is particularly suitable for scenarios such as intelligent connected vehicle monitoring and commercial vehicle monitoring that have high requirements for real-time and security.

[0005] To solve the above technical problems, the first aspect of the present invention discloses a method for remotely monitoring vehicle video, which is applied to a cloud server; wherein the cloud server includes an independent audio and video cloud server and a vehicle control cloud server; the method comprises:

[0006] When a mobile terminal has a video viewing demand, a function start command actively issued by the mobile terminal is transmitted to the vehicle bound to the mobile terminal via a control link to wake up the vehicle; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle;

[0007] Transmitting video control parameters actively sent by the mobile terminal to the vehicle via a video transmission link, so that the vehicle captures a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least an audio and video cloud server and the vehicle;

[0008] The target video segment is transmitted to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmits the target video segment to the mobile terminal for viewing.

[0009] Optionally, before transmitting the video control parameters actively sent by the mobile terminal to the vehicle via the video transmission link, the method further includes:

[0010] The audio and video cloud server sends a user verification request to the vehicle control cloud server, so that the vehicle control cloud server authenticates the audio and video cloud server.

[0011] Optionally, transmitting the target video clip to the audio and video cloud server through the video transmission link, and transmitting the target video clip to the mobile terminal for viewing by the audio and video cloud server, specifically includes:

[0012] The target video clip is transmitted to the audio and video cloud server through the video transmission link, and the function execution status signal corresponding to the target video clip is synchronously transmitted to the vehicle control cloud server through the video transmission link, and the audio and video cloud server and the vehicle control cloud server respectively transmit the target video clip and the function execution status signal to the mobile terminal.

[0013] Optionally, after transmitting the target video clip to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmitting the target video clip to the mobile terminal for viewing, the method further includes:

[0014] The audio stream actively sent by the mobile terminal is transmitted to the vehicle through the video transmission link, so that the vehicle plays the audio stream.

[0015] Optionally, the method further includes:

[0016] When an emergency occurs in the vehicle, the abnormal vehicle environment alarm signal uploaded by the vehicle is transmitted to the vehicle control cloud server via the control link, and the abnormal video clip uploaded by the vehicle is transmitted to the audio and video cloud server for caching via the video transmission link;

[0017] The vehicle control cloud server pushes an alarm message to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein, the alarm message includes the video cache address of the audio and video cloud server that caches the abnormal video clip, and the video cache address is sent by the audio and video cloud server to the vehicle control cloud server.

[0018] A second aspect of the present invention discloses a method for remotely monitoring video inside and outside a vehicle, the method being applied to a vehicle and comprising:

[0019] When a mobile terminal bound to the vehicle has a video viewing demand, the mobile terminal receives a function start command actively issued by the mobile terminal and transmitted via a control link, and performs a wake-up operation according to the function start command; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle;

[0020] receiving video control parameters actively sent by the mobile terminal and transmitted via a video transmission link, and capturing a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least an audio and video cloud server and the vehicle;

[0021] The target video segment is transmitted to the audio and video cloud server via the video transmission link, so that the audio and video cloud server forwards the target video segment to the mobile terminal for viewing.

[0022] Optionally, the method further includes:

[0023] When an emergency occurs in the vehicle, an abnormal vehicle environment alarm signal is generated, and the abnormal video clip is captured according to the abnormal vehicle environment alarm signal;

[0024] The abnormal video clip is transmitted to the audio and video cloud server through the video transmission link for caching, so that the audio and video cloud server sends the video cache address of the abnormal video clip to the vehicle control cloud server; and the vehicle environment abnormality alarm signal is transmitted to the vehicle control cloud server through the control link, so that the vehicle control cloud server pushes alarm information to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein the alarm information includes the video cache address.

[0025] A third aspect of the present invention discloses a cloud server, which includes an audio and video cloud server module and a vehicle control cloud server module that are independent of each other; wherein,

[0026] The vehicle control cloud server module is used to transmit a function start command actively issued by the mobile terminal to the vehicle bound to the mobile terminal via a control link when the mobile terminal has a video viewing demand, so as to wake up the vehicle; wherein the control link is a control parameter transmission link composed of at least the vehicle control cloud server module and the vehicle;

[0027] The audio and video cloud server module is used to transmit the video control parameters actively sent by the mobile terminal to the vehicle via a video transmission link, so that the vehicle can shoot a target video clip according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least the audio and video cloud server module and the vehicle;

[0028] The audio and video cloud server module is used to receive the target video segment transmitted by the video transmission link, and transmit the target video segment to the mobile terminal for viewing.

[0029] Optionally, the audio and video cloud server module is used to send a user verification request to the vehicle control cloud server module, so that the vehicle control cloud server module authenticates the audio and video cloud server module.

[0030] Optionally, the audio and video cloud server module is used to receive the target video segment transmitted by the video transmission link;

[0031] The vehicle control cloud server module is configured to receive a function execution status signal corresponding to the target video segment synchronously transmitted by the control link;

[0032] The audio and video cloud server module and the vehicle control cloud server module are also used to transmit the target video clip and the function execution status signal to the mobile terminal respectively.

[0033] Optionally, the audio and video cloud server module is used to transmit the audio stream actively sent by the mobile terminal to the vehicle through the video transmission link, so that the vehicle can play the audio stream.

[0034] Optionally, when an emergency occurs in the vehicle, the vehicle control cloud server module is further configured to receive a vehicle environment abnormality alarm signal transmitted by the control link and uploaded by the vehicle, and the audio and video cloud server module is further configured to receive and cache abnormal video clips transmitted by the video transmission link and shot and uploaded by the vehicle;

[0035] The vehicle control cloud server module is also used to push alarm information to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein, the alarm information includes the video cache address of the abnormal video clip cached by the audio and video cloud server module, and the video cache address is sent by the audio and video cloud server module to the vehicle control cloud server module.

[0036] A fourth aspect of the present invention discloses a vehicle, comprising:

[0037] a first receiving unit configured to receive a function activation command proactively issued by a mobile terminal bound to the vehicle and transmitted via a control link when the mobile terminal has a video viewing requirement, and to perform a wake-up operation according to the function activation command; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle;

[0038] a second receiving unit, configured to receive video control parameters proactively sent by the mobile terminal and transmitted via a video transmission link, and capture a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link consisting of at least an audio and video cloud server and the vehicle;

[0039] The sending unit is used to transmit the target video segment to the audio and video cloud server through the video transmission link, so that the audio and video cloud server forwards the target video segment to the mobile terminal for viewing.

[0040] Optionally, the vehicle further includes:

[0041] an abnormality monitoring unit, configured to generate a vehicle environment abnormality alarm signal when an emergency occurs to the vehicle, and trigger the capture of abnormal video clips according to the vehicle environment abnormality alarm signal;

[0042] The sending unit is also used to transmit the abnormal video clip to the audio and video cloud server through the video transmission link for caching, so that the audio and video cloud server sends the video cache address of the abnormal video clip to the vehicle control cloud server; and transmit the vehicle environment abnormality alarm signal to the vehicle control cloud server through the control link, so that the vehicle control cloud server pushes alarm information to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein the alarm information includes the video cache address.

[0043] A fifth aspect of the present invention discloses a system for remotely monitoring video inside and outside a vehicle, the system comprising: a mobile terminal, a cloud server, and a vehicle; the cloud server comprises an independent audio and video cloud server and a vehicle control cloud server; wherein,

[0044] The mobile terminal is used to actively send function start commands and video control parameters when there is a need to view a video;

[0045] The cloud server is configured to transmit the function activation command to the vehicle bound to the mobile terminal via a control link, and to transmit the video control parameters to the vehicle via a video transmission link; wherein the control link is a control parameter transmission link composed of at least the vehicle control cloud server and the vehicle; and the video transmission link is a multimedia data transmission link composed of at least the audio and video cloud server and the vehicle;

[0046] The vehicle is configured to receive the function start command and perform a wake-up operation according to the function start command; and receive the video control parameter and shoot a target video clip according to the video control parameter;

[0047] The cloud server is further configured to receive the target video segment transmitted by the video transmission link, and transmit the target video segment to the mobile terminal for viewing.

[0048] Through one or more technical solutions of the present invention, the present invention has the following beneficial effects or advantages:

[0049] The present invention provides a method, cloud server, vehicle, and system for remotely monitoring videos inside and outside a vehicle. Different links are used to transmit control commands and video data, which can meet the communication requirements of high real-time performance and low latency for vehicle body control, as well as the requirements of high bandwidth and high network load for video transmission, thereby significantly improving the real-time performance and security of monitoring.

[0050] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0052] Figure 1 The overall architecture of a system for remotely monitoring vehicle video according to one embodiment of the present invention is shown;

[0053] Figure 2A flow chart of a method for remotely monitoring vehicle video on a cloud server side according to an embodiment of the present invention is shown;

[0054] Figure 3 A flow chart of a method for remotely monitoring vehicle video on a vehicle side according to one embodiment of the present invention is shown;

[0055] Figure 4 It shows an overall control logic diagram of a user actively viewing a video around a vehicle according to one embodiment of the present invention;

[0056] Figure 5 It shows the overall control logic diagram of a vehicle actively alerting and uploading a video of the vehicle's surroundings according to one embodiment of the present invention;

[0057] Figure 6 shows an architecture diagram of a cloud server according to one embodiment of the present invention;

[0058] Figure 7 A vehicle architecture diagram according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0060] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0061] In a first aspect, an embodiment of the present invention provides a method for remotely monitoring vehicle video, which is applied to a cloud server side and a vehicle side in a system for remotely monitoring vehicle video.

[0062] See Figure 1 , is the overall architecture diagram of the remote monitoring vehicle video system, including mobile terminals, cloud servers and vehicles.

[0063] Among them, the mobile terminal includes but is not limited to mobile phones, PADs (Portable Android Devices), computers and other electronic devices. Taking a mobile phone as an example, the body control APP (Application Program) installed on the mobile phone provides a vehicle alarm prompt function and an HMI (Human-Machine Interface) interface for remotely viewing the video around the vehicle. If the APP receives an alarm message pushed by the vehicle control cloud server, it will notify the owner of the vehicle alarm prompt through ringing and / or pop-up windows. The user opens the vehicle video viewing function icon on the mobile phone APP and switches to the HMI interface. The HMI interface includes a video playback window, a perspective selection button, and a voice call button. Use the perspective selection button to switch to view the 360-degree surround view outside the car or the perspective inside the car. In addition, the mobile phone APP obtains the vehicle-side video from the audio and video cloud server and plays it in the video playback window. The user clicks the voice call button to turn on or off the call function with the outside of the car.

[0064] The cloud server includes an independent audio and video cloud server and a vehicle control cloud server. In this embodiment, the vehicle control cloud server is primarily used to manage and control functional permissions between the mobile terminal and the vehicle. The audio and video cloud server is used to store or convert multimedia data.

[0065] It is worth noting that although the audio and video cloud server and the vehicle control cloud server are physically independent of each other, they are both private clouds independently developed by the enterprise. The two can be in the same enterprise intranet and work together to form a closed loop. Both are protected by the internal enterprise firewall, and the internal transmission security mechanism is relatively high. It can be seen that the audio and video cloud server of this application is significantly different from the third-party server. In addition, the audio and video cloud server of this application is also different from the live broadcast server. Although the live broadcast server can also broadcast and send vehicle-type videos directly, it needs to build a room number to push the vehicle-type live broadcast video in a one-way manner. The audio and video cloud server and vehicle of this application can form a video transmission link and support two-way voice calls between the vehicle and the mobile terminal. It can transmit audio streams in the direction of vehicle-audio and video cloud server-mobile terminal, and can also transmit audio streams in the direction of mobile terminal-audio and video cloud server-vehicle terminal.

[0066] In this embodiment, the vehicle includes a telematics box (TBOX), a body control module (BCM), an in-vehicle infotainment (IVI), an around view monitor (AVM), four exterior surround view cameras 11, one interior camera 12, one exterior microphone 13, and one exterior speaker 14.

[0067] The connected vehicle terminal (TBOX), body controller (BCM), cockpit domain controller (IVI), and surround view controller (AVM) are connected via the CAN bus. These terminals support network management (NM) and local network management (PN). Four exterior surround view cameras (11) are connected to the AVM. The AVM and IVI are connected via an LVDS bus, transmitting 360-degree surround view video to the IVI. The interior camera (12) is connected to the IVI via the LVDS bus. The exterior microphone (13) and exterior speaker (14) are connected to the IVI via the A2B bus.

[0068] Remote monitoring vehicle video method on the cloud server side, such as Figure 2 As shown, the following steps are included:

[0069] S201, when the mobile terminal has a video viewing demand, a function start command actively issued by the mobile terminal is transmitted to the vehicle bound to the mobile terminal through the control link to wake up the vehicle.

[0070] Among them, the control link is a control parameter transmission link composed of at least a vehicle control cloud server and a vehicle.

[0071] Furthermore, when the mobile terminal has a video viewing requirement, the function start command and perspective setting command actively issued by the mobile terminal are transmitted to the vehicle bound to the mobile terminal through the control link; among them, the function start command is used to wake up the vehicle; the perspective setting command is used to determine the target camera from the cameras inside and outside the vehicle to shoot the target video clip that meets the perspective.

[0072] Specifically, the mobile HMI interface includes a video playback window, a view selection button, and a voice call button. The view selection buttons include: interior view, left front view, left rear view, right front view, right rear view, and panoramic view. Selecting a view selection button generates a view setting command. This command determines the target camera for the final shot.

[0073] For the cloud server, the function start command and the perspective setting command are received through the vehicle control cloud server in the cloud server, and the function start command is transmitted to the vehicle bound to the mobile terminal through the control link composed of the vehicle control cloud server and the vehicle to wake up the vehicle.

[0074] S202: Transmitting the video control parameters actively sent by the mobile terminal to the vehicle through the video transmission link, so that the vehicle shoots the target video clip according to the video control parameters.

[0075] Among them, the video transmission link is a multimedia data transmission link composed of at least an audio and video cloud server and a vehicle.

[0076] In an optional embodiment, to verify the legitimacy of the video transmission link, before transmitting the video control parameters proactively issued by the mobile terminal to the vehicle via the video transmission link, the audio and video cloud server sends a user verification request to the vehicle control cloud server, allowing the vehicle control cloud server to authenticate the audio and video cloud server. The user verification request carries the user ID. If the authentication is successful, the video transmission link is deemed legitimate, allowing the video control parameters to be transmitted to the vehicle via the video transmission link.

[0077] When a mobile device requests video viewing, in addition to generating a function activation command and a viewing angle setting command, it also generates video control parameters. These parameters include the video request, the terminal's UID (User Identifier), the transmission protocol, the communication port number, and more. These parameters are only used to request the upload of the target video clip and do not restrict the shooting angle of the target video clip.

[0078] For the cloud server, the video control parameters are received through the audio and video cloud server in the cloud server, and then the video control parameters are transmitted to the vehicle through the video transmission link.

[0079] In an optional embodiment, when shooting the target video clip, the microphone in the vehicle is controlled to synchronously capture the target audio. The target audio can be audio generated around or inside the vehicle. Optionally, when receiving the target video clip through the video transmission link, the target video clip and the target audio are synchronously received through the video transmission link, the target video clip and the target audio are fused in the audio and video cloud server, and transmitted to the mobile terminal for playback. Optionally, the vehicle side can also synchronously capture the target audio and fuse it into the target video clip, and then synchronously receive the fused target video clip through the video transmission link, and transmit it to the mobile terminal for playback.

[0080] In one optional embodiment, when capturing a target video clip, the vehicle simultaneously generates a function execution status signal or a function start failure reason signal. Specifically, after transmitting video control parameters proactively issued by the mobile terminal to the vehicle via a video transmission link, if the vehicle successfully captures the target video clip according to the video control parameters, a function execution status signal corresponding to the target video clip is simultaneously generated. If a malfunction prevents the vehicle from successfully capturing the target video clip, a function start failure reason signal is simultaneously generated.

[0081] Function execution status signals include:

[0082] 0x0: Disable the function.

[0083] 0x1: System initialization.

[0084] 0x2: The in-car video stream is being pushed.

[0085] 0x3: The external video stream is being pushed.

[0086] 0x4: Function activation failed.

[0087] 0x5 to 0x7: Reserved.

[0088] Function startup failure reason signals include:

[0089] 1x0: No fault.

[0090] 1x1: The vehicle camera failed to start.

[0091] 1x2: Video stream upload failed.

[0092] 1x3: Other failures.

[0093] 1x4 to 1x7: Reserved.

[0094] S203, transmitting the target video clip to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmits the target video clip to the mobile terminal for viewing.

[0095] In an optional embodiment, if the target video clip is successfully received, it means that the vehicle has successfully captured the target video clip according to the video control parameter control. At this time, the target video clip is transmitted to the audio and video cloud server via the video transmission link, and the function execution status signal corresponding to the target video clip is synchronously transmitted to the vehicle control cloud server via the control link. The audio and video cloud server and the vehicle control cloud server respectively transmit the target video clip and the function execution status signal to the mobile terminal. If the target video clip cannot be received via the video transmission link, it means that the vehicle cannot successfully capture the target video clip due to a fault. At this time, a function start failure reason signal is received to the vehicle control cloud server via the control link, and the vehicle control cloud server transmits the function start failure reason signal to the mobile terminal.

[0096] In an optional embodiment, in order to support the user's voice being played out loud around the vehicle, after the target video clip is transmitted to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmits the target video clip to the mobile terminal for viewing, the audio stream actively sent by the mobile terminal is transmitted to the vehicle via the video transmission link, so that the vehicle can play the audio stream out loud, which facilitates communication between the user and people around the vehicle.

[0097] The above is the control logic executed on the cloud server side for users to actively view the video around the vehicle. This solution supports users to switch between internal and external perspectives of vehicle videos remotely in real time, providing a richer driving experience.

[0098] The following describes the control logic for the vehicle to trigger an alarm to the user when an emergency occurs.

[0099] When an emergency occurs in the vehicle, such as when the vehicle is awakened by a built-in vibration sensor (being bumped) or detects that a door has been abnormally opened, the vehicle environment abnormality alarm signal uploaded by the vehicle is transmitted to the vehicle control cloud server via the control link, and the abnormal video clips uploaded by the vehicle are transmitted to the audio and video cloud server via the video transmission link for caching. Among them, the abnormal video clips can be intercepted according to the set time before and after the vehicle emergency. For example, the abnormal video clips 10 seconds before and 5 seconds after the vehicle environment abnormality alarm signal are intercepted from the vehicle cache and sent to the audio and video cloud server.

[0100] For the audio and video cloud server, after caching the abnormal video clip, the video cache address of the cached abnormal video clip is sent to the vehicle control cloud server.

[0101] The vehicle control cloud server pushes an alarm message to the mobile terminal based on the abnormal vehicle environment alarm signal. The alarm message includes the video cache address of the abnormal video clip cached by the audio and video cloud server, which is sent by the audio and video cloud server to the vehicle control cloud server.

[0102] After receiving an alert message from the vehicle control cloud server, the mobile device displays a pop-up window with a download and delete button. If the user clicks the download button, the mobile device downloads the video from the video cloud server according to the video cache address. If the user clicks the delete button, the mobile device sends a delete command to the video cloud server, instructing it to delete the cached abnormal video clip.

[0103] Of course, to support the user's voice being played out loud in the vehicle's surroundings, the mobile device generates an audio stream. The vehicle control cloud server in the cloud server receives the audio stream and transmits the audio stream actively sent by the mobile device to the vehicle via a video transmission link, allowing the vehicle to play the audio stream out loud, facilitating communication between the user and those around the vehicle.

[0104] The above is the control logic for proactively alerting users to view videos around the vehicle when an abnormality occurs. When an emergency occurs with the vehicle, such as when the vehicle is bumped or illegally entered, users can be promptly alerted and provided with abnormal video footage of the incident for users to manage losses and subsequently investigate liability.

[0105] The above technical solution describes the overall control logic for remotely monitoring videos inside and outside the vehicle in a cloud server. By deploying the functional control service and video transmission management service in different cloud servers respectively, and using different links for transmission of control instructions and video data, it can not only meet the real-time and low-latency communication requirements of vehicle body control, but also meet the high-bandwidth and high-network load requirements of video transmission, thereby significantly improving the real-time and security of monitoring.

[0106] In the second aspect, based on the same inventive concept as the first aspect, the embodiment of the present invention further provides a method for remotely monitoring the video inside and outside the vehicle, which is applied to the vehicle side, see Figure 3 , the method comprises the following steps:

[0107] S301, when the mobile terminal bound to the vehicle has a video viewing requirement, it receives a function start command actively sent by the mobile terminal and transmitted through the control link, and performs a wake-up operation according to the function start command.

[0108] Among them, the control link is a control parameter transmission link composed of at least a vehicle control cloud server and a vehicle.

[0109] Specifically, when the mobile terminal bound to the vehicle has a video viewing requirement, it receives the function start command and view setting command actively issued by the mobile terminal and transmitted through the control link, and performs the wake-up operation according to the function start command, and controls the corresponding target camera to shoot according to the view setting command.

[0110] Furthermore, the vehicle networking terminal TBOX in the vehicle receives the function start command and performs a wake-up operation. Specifically, the vehicle networking terminal TBOX wakes up the vehicle by waking up the CAN network.

[0111] In this embodiment, after the vehicle wakes up, the cockpit domain controller IVI receives a function start command and a view angle setting command from the vehicle networking terminal TBOX.

[0112] Furthermore, after receiving the function start command from the Internet of Vehicles terminal TBOX, the cockpit domain controller IVI in the vehicle establishes a video transmission link with the audio and video cloud server; then, based on the perspective setting command, it determines the target camera from the cameras inside and outside the vehicle and shoots the target video clip. Specifically, if the video setting command is an in-vehicle perspective setting command, the in-vehicle camera is used as the target camera to shoot the corresponding target video clip inside the vehicle. If the perspective setting command is a panoramic perspective setting command, a video extraction request is sent to the surround view controller AVM to control the surround view controller AVM to drive the four surround view cameras to shoot the target video clip. After splicing, compression, and encoding, the video clip is transmitted to the cockpit domain controller IVI via the vehicle Ethernet.

[0113] S302: Receive video control parameters actively sent by the mobile terminal and transmitted through the video transmission link, and shoot a target video segment according to the video control parameters.

[0114] Among them, the video transmission link is a multimedia data transmission link composed of at least an audio and video cloud server and a vehicle.

[0115] In specific implementations, the IVI receives video control parameters from the connected vehicle (TBOX) terminal and triggers the target camera to capture the target video clip. The target video clip includes an image. For example, the IVI directly controls the in-vehicle camera to capture the target video clip, or sends a video extraction request to the AVM to control the AVM to drive the four surround view cameras to capture and process the target video clip.

[0116] In one optional embodiment, while capturing a target video clip, the vehicle simultaneously generates a function execution status signal or a function activation failure reason signal. Specifically, if the vehicle successfully captures the target video clip according to the video control parameters, a function execution status signal corresponding to the target video clip is simultaneously generated. If a malfunction prevents the vehicle from successfully capturing the target video clip, a function activation failure reason signal is simultaneously generated.

[0117] S303: Transmit the target video clip to the audio and video cloud server through the video transmission link, so that the audio and video cloud server forwards the target video clip to the mobile terminal for viewing.

[0118] While uploading the target video clip, the function execution status signal or function startup failure reason signal corresponding to the target video clip is synchronously transmitted to the vehicle control cloud server through the control link, so that the vehicle control cloud server forwards the function execution status signal or function startup failure reason signal to the mobile terminal for viewing.

[0119] In an optional embodiment, in order to support the user's voice being played outward around the vehicle, after the target video clip is transmitted to the audio and video cloud server via the video transmission link, the audio stream actively sent by the mobile terminal and transmitted by the video transmission link is received and played outward, which facilitates communication between the user and people around the vehicle.

[0120] In an optional embodiment, when shooting the target video clip, the microphone in the vehicle is controlled to synchronously collect the target audio. The target audio can be audio generated around or inside the vehicle. Optionally, when the target video clip is transmitted to the audio and video cloud server via the video transmission link, the target video clip and the target audio are synchronously transmitted to the audio and video cloud server via the video transmission link, and the target video clip and the target audio are fused in the audio and video cloud server and transmitted to the mobile terminal for playback. Optionally, the vehicle side can also synchronously collect the target audio and fuse it to the target video clip, and then synchronously transmit the fused target video clip to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmits it to the mobile terminal for playback.

[0121] The above is the control logic executed on the vehicle side for users to actively view the video around the vehicle. This solution supports users to switch between internal and external perspectives of vehicle videos remotely in real time, providing a richer car experience.

[0122] The following describes the control logic for the vehicle to trigger an alarm to the user when an emergency occurs.

[0123] When an emergency occurs in the vehicle, such as when the vehicle is awakened by a built-in vibration sensor (being bumped) or when a door is detected to be abnormally opened, a vehicle environment abnormality alarm signal is generated, and the abnormality video clip is captured based on the vehicle environment abnormality alarm signal. Among them, the abnormal video clip can be intercepted according to the set time before and after the vehicle emergency occurs, for example, the abnormal video clip 10 seconds before and 5 seconds after the vehicle environment abnormality alarm signal is received is captured from the vehicle cache.

[0124] The abnormal video clip is transmitted to the audio and video cloud server through the video transmission link for caching, so that the audio and video cloud server sends the video cache address of the cached abnormal video clip to the vehicle control cloud server; and the vehicle environment abnormality alarm signal is transmitted to the vehicle control cloud server through the control link, so that the vehicle control cloud server pushes the alarm information to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein the alarm information includes the video cache address.

[0125] After receiving an alert message from the vehicle control cloud server, the mobile device displays a pop-up window with a download and delete button. If the user clicks the download button, the mobile device downloads the video from the video cloud server according to the video cache address. If the user clicks the delete button, the mobile device sends a delete command to the video cloud server, instructing it to delete the cached abnormal video clip.

[0126] Of course, in order to support the user's voice being played out loud around the vehicle, the vehicle side receives the audio stream actively sent by the mobile terminal through the video transmission link and plays it out loud, making it easier for users to communicate with people around the vehicle.

[0127] The above is the control logic for proactively alerting users to view videos around the vehicle when an abnormality occurs. When an emergency occurs with the vehicle, such as when the vehicle is bumped or illegally entered, users can be promptly alerted and provided with abnormal video footage of the incident for users to manage losses and subsequently investigate liability.

[0128] This technical solution describes the overall control logic for remotely monitoring the internal and external videos of the vehicle on the vehicle side. Different links are used for transmission of control instructions and video data. This can meet both the real-time and low-latency communication requirements of vehicle body control and the high-bandwidth and high-network load requirements of video transmission, thereby significantly improving the real-time and security of monitoring.

[0129] In addition, through the collaboration between the Internet of Vehicles terminal TBOX, the body controller BCM, the cockpit domain controller IVI, and the surround view controller AVM, abnormal situations can be detected and abnormal video clips can be captured in a timely manner. There is no need to install an autonomous driving system, only a simple surround view control system is required. It is lower in cost than similar video surveillance solutions and is applicable to a wider range of vehicle models.

[0130] In actual use, in order to better understand the two remote video viewing methods, see the following Figure 4-Figure 5 .in, Figure 4 It is the overall control logic for users to actively view the video around the vehicle. Figure 5 It is the overall control logic for the vehicle to actively issue an alarm and upload the video around the vehicle.

[0131] The overall control logic for users to actively view the video around the vehicle is as follows:

[0132] The user clicks the remote viewing vehicle video function icon on the mobile phone APP to open the function page, checks the outdoor panoramic view or indoor view option on the page, and then clicks the view button.

[0133] The mobile phone APP generates a viewing angle setting command based on the user's selected options, sends function start commands and viewing angle setting commands to the vehicle control cloud server, and sends video control parameters (video request, terminal UID, transmission protocol, communication port number, etc.) to the audio and video cloud server.

[0134] The audio and video cloud server initiates a user verification request to the vehicle control cloud server, so that the vehicle control cloud server can authenticate (user authority verification) and control functions of the audio and video cloud server.

[0135] After receiving the function start command and the viewing angle setting command, the vehicle control cloud server sends the function start command and the viewing angle setting command to the vehicle through the control link.

[0136] After the audio and video cloud server passes the authentication, the video control parameters are sent to the vehicle through the video transmission link.

[0137] After receiving the cloud-based function start command, the vehicle's TBOX terminal determines the CAN network status. If it is dormant, it first sends a network management message to wake up the network, then sends the function start command and view angle setting command to the cockpit domain controller (IVI). If the network is already awake, it directly sends the function start command and view angle setting command to the cockpit domain controller (IVI).

[0138] The cockpit domain controller IVI receives the function start command from the Internet of Vehicles terminal TBOX, establishes a communication connection with the audio and video cloud server, and determines the target camera based on the perspective setting command transmitted by the Internet of Vehicles terminal TBOX.

[0139] If the video setting command is for setting the in-vehicle perspective, the in-vehicle camera is used as the target camera to capture the in-vehicle video stream, thereby obtaining the target video clip. If the perspective setting command is for setting the panoramic perspective, a video extraction request is sent to the surround view controller (AVM) to control the surround view controller (AVM) to drive the four surround view cameras to capture the external surround view video stream. After splicing, compressing, and encoding, the target video clip is obtained and transmitted to the cockpit domain controller (IVI) via the in-vehicle Ethernet.

[0140] After the cockpit domain controller IVI successfully starts the in-car camera to capture the target video clip, or receives the target video clip from the surround view controller AVM, it pushes the target video clip to the audio and video cloud server through the video transmission link.

[0141] During the process of starting the functions of the cockpit domain controller IVI and the surround view controller AVM and executing shooting, the cockpit domain controller IVI sends a function execution status signal or a function start failure reason signal to the vehicle network terminal TBOX.

[0142] The audio and video cloud server receives the target video clip uploaded by the cockpit domain controller IVI through the video transmission link and pushes it to the mobile phone APP.

[0143] The vehicle control cloud server synchronously receives the function execution status signal or function startup failure reason signal forwarded by the Internet of Vehicles terminal TBOX through the control link and transmits it to the mobile phone APP.

[0144] The mobile phone APP decodes and displays the target video clip, or prompts the user based on the signal of the reason for the function startup failure sent by the vehicle control cloud server.

[0145] The overall control logic for the vehicle to proactively issue an alert and upload surrounding video is as follows:

[0146] When the body controller BCM is dormant in the vehicle CAN network, it is awakened by the built-in vibration sensor or detects that the door is abnormally opened. It then wakes up the vehicle network and sends a vehicle environment abnormality alarm signal to the cockpit domain controller IVI, surround view controller AVM, and cockpit domain controller TBOX.

[0147] The Internet of Vehicles terminal TBOX receives the vehicle environment abnormality alarm signal and sends the vehicle environment abnormality alarm signal to the vehicle control cloud server through the control link.

[0148] The cockpit domain controller (IVI) receives an abnormal vehicle environment alarm signal and sends an abnormal outside vehicle video extraction command to the surround view controller (AVM). The surround view controller (AVM) is controlled to intercept the 10 seconds before and 5 seconds after receiving the signal from the cache as the abnormal video stream, and establish a connection with the audio and video cloud. After the abnormal video clip is desensitized, it is sent to the audio and video cloud server via the video transmission link.

[0149] The audio and video cloud server caches the video files sent by the cockpit domain controller IVI and sends the video cache address to the vehicle control cloud server.

[0150] After the vehicle control cloud server receives the abnormal vehicle environment alarm signal reported by the Internet of Vehicles terminal TBOX through the control link, it generates an alarm message and pushes it to the mobile phone APP. The alarm message contains the video cache address.

[0151] When the mobile phone APP receives the alarm message pushed by the vehicle control cloud server, a pop-up window will be displayed to remind the user, which contains a download button and a delete button.

[0152] If the user clicks the download button, the mobile app requests the video to be downloaded from the video cloud server. The video cloud server receives the download request and sends the video file to the mobile app.

[0153] If the user clicks the delete button, the mobile app sends a delete video command to the audio and video cloud server. Upon receiving the delete request, the audio and video cloud server deletes the cached abnormal video clips.

[0154] Of course, users can also use the mobile phone APP to send video control parameters to control the vehicle to shoot the required video.

[0155] In the third aspect, based on the same inventive concept as the method for remotely monitoring the internal and external video of a vehicle provided in the embodiment of the first aspect, the embodiment of the present invention discloses a cloud server, see Figure 6 The cloud server includes an independent audio and video cloud server module 601 and a vehicle control cloud server module 602; wherein,

[0156] The vehicle control cloud server module 602 is used to transmit a function start command actively issued by the mobile terminal to the vehicle bound to the mobile terminal via a control link when the mobile terminal has a video viewing demand, so as to wake up the vehicle; wherein the control link is a control parameter transmission link composed of at least the vehicle control cloud server module 602 and the vehicle;

[0157] The audio and video cloud server module 601 is used to transmit the video control parameters actively sent by the mobile terminal to the vehicle via a video transmission link, so that the vehicle can shoot a target video clip according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least the audio and video cloud server module 601 and the vehicle;

[0158] The audio and video cloud server module 601 is used to receive the target video segment transmitted by the video transmission link, and transmit the target video segment to the mobile terminal for viewing.

[0159] It should be noted that the cloud server provided in the embodiment of the present invention, wherein the specific manner in which each part performs operations has been described in detail in the method embodiment provided in the above-mentioned first aspect. The specific implementation process can refer to the method embodiment provided in the above-mentioned first aspect, and will not be elaborated on here.

[0160] In a fourth aspect, based on the same inventive concept as the method for remotely monitoring the internal and external video of a vehicle provided in the embodiment of the first aspect, an embodiment of the present invention discloses a vehicle, Figure 7 ,include:

[0161] The first receiving unit 701 is configured to receive a function start command proactively issued by the mobile terminal bound to the vehicle and transmitted via a control link when the mobile terminal has a video viewing requirement, and to perform a wake-up operation according to the function start command; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle;

[0162] A second receiving unit 702 is configured to receive video control parameters proactively sent by the mobile terminal and transmitted via a video transmission link, and capture a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link consisting of at least an audio and video cloud server and the vehicle;

[0163] The sending unit 703 is configured to transmit the target video segment to the audio and video cloud server via the video transmission link, so that the audio and video cloud server forwards the target video segment to the mobile terminal for viewing.

[0164] It should be noted that the specific manner in which each part of the vehicle provided in the embodiment of the present invention performs operations has been described in detail in the method embodiment provided in the above-mentioned second aspect. The specific implementation process can refer to the method embodiment provided in the above-mentioned second aspect and will not be elaborated on here.

[0165] In a fifth aspect, based on the same inventive concept as the method for remotely monitoring the video inside and outside a vehicle provided in the embodiment of the first aspect, an embodiment of the present invention discloses a system for remotely monitoring the video inside and outside a vehicle, such as Figure 1 As shown, the system includes: a mobile terminal, a cloud server and a vehicle; wherein the cloud server includes an independent audio and video cloud server and a vehicle control cloud server; wherein,

[0166] The mobile terminal is used to actively send function start commands and video control parameters when there is a need to view a video;

[0167] The cloud server is configured to transmit the function activation command to the vehicle bound to the mobile terminal via a control link, and to transmit the video control parameters to the vehicle via a video transmission link; wherein the control link is a control parameter transmission link composed of at least the vehicle control cloud server and the vehicle; and the video transmission link is a multimedia data transmission link composed of at least the audio and video cloud server and the vehicle;

[0168] The vehicle is configured to receive the function start command and perform a wake-up operation according to the function start command; and receive the video control parameter and shoot a target video clip according to the video control parameter;

[0169] The cloud server is further configured to receive the target video segment via the video transmission link and transmit the target video segment to the mobile terminal for viewing.

[0170] It should be noted that the system for remotely monitoring videos inside and outside a vehicle provided by an embodiment of the present invention, wherein the specific manner in which each part performs operations has been described in detail in the method embodiments provided in the above-mentioned first and second aspects. The specific implementation process can refer to the method embodiments provided in the above-mentioned first and second aspects, and will not be elaborated on here.

[0171] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0172] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for remotely monitoring vehicle video, characterized in that: The method is applied to a cloud server; wherein the cloud server includes an audio and video cloud server and a vehicle control cloud server that are independent of each other; the method includes: When a mobile terminal has a video viewing demand, a function start command actively issued by the mobile terminal is transmitted to the vehicle bound to the mobile terminal via a control link to wake up the vehicle; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle; Transmitting video control parameters actively sent by the mobile terminal to the vehicle via a video transmission link, so that the vehicle captures a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least an audio and video cloud server and the vehicle; The target video segment is transmitted to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmits the target video segment to the mobile terminal for viewing.

2. The method according to claim 1, wherein Before transmitting the video control parameters actively sent by the mobile terminal to the vehicle through the video transmission link, the method further includes: The audio and video cloud server sends a user verification request to the vehicle control cloud server, so that the vehicle control cloud server authenticates the audio and video cloud server.

3. The method according to claim 1, wherein The step of transmitting the target video clip to the audio and video cloud server through the video transmission link, and transmitting the target video clip to the mobile terminal for viewing by the audio and video cloud server, specifically includes: The target video clip is transmitted to the audio and video cloud server through the video transmission link, and the function execution status signal corresponding to the target video clip is synchronously transmitted to the vehicle control cloud server through the control link, and the audio and video cloud server and the vehicle control cloud server respectively transmit the target video clip and the function execution status signal to the mobile terminal.

4. The method according to claim 1, wherein After transmitting the target video clip to the audio and video cloud server via the video transmission link, and the audio and video cloud server transmitting the target video clip to the mobile terminal for viewing, the method further includes: The audio stream actively sent by the mobile terminal is transmitted to the vehicle through the video transmission link, so that the vehicle plays the audio stream.

5. The method according to claim 1, wherein The method further comprises: When an emergency occurs in the vehicle, the abnormal vehicle environment alarm signal uploaded by the vehicle is transmitted to the vehicle control cloud server via the control link, and the abnormal video clip uploaded by the vehicle is transmitted to the audio and video cloud server for caching via the video transmission link; The vehicle control cloud server pushes an alarm message to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein, the alarm message includes the video cache address of the audio and video cloud server that caches the abnormal video clip, and the video cache address is sent by the audio and video cloud server to the vehicle control cloud server.

6. A method for remotely monitoring video inside and outside a vehicle, characterized in that: The method is applied to a vehicle, and comprises: When a mobile terminal bound to the vehicle has a video viewing demand, the mobile terminal receives a function start command actively issued by the mobile terminal and transmitted via a control link, and performs a wake-up operation according to the function start command; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle; receiving video control parameters actively sent by the mobile terminal and transmitted via a video transmission link, and capturing a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least an audio and video cloud server and the vehicle; The target video segment is transmitted to the audio and video cloud server via the video transmission link, so that the audio and video cloud server forwards the target video segment to the mobile terminal for viewing.

7. The method according to claim 6, wherein The method further comprises: When an emergency occurs in the vehicle, an abnormal vehicle environment alarm signal is generated, and the abnormal video clip is captured according to the abnormal vehicle environment alarm signal; The abnormal video clip is transmitted to the audio and video cloud server through the video transmission link for caching, so that the audio and video cloud server sends the video cache address of the abnormal video clip to the vehicle control cloud server; and the vehicle environment abnormality alarm signal is transmitted to the vehicle control cloud server through the control link, so that the vehicle control cloud server pushes alarm information to the mobile terminal according to the vehicle environment abnormality alarm signal; wherein the alarm information includes the video cache address.

8. A cloud server, characterized in that: The cloud server includes an independent audio and video cloud server module and a vehicle control cloud server module; wherein, The vehicle control cloud server module is used to transmit a function start command actively issued by the mobile terminal to the vehicle bound to the mobile terminal via a control link when the mobile terminal has a video viewing demand, so as to wake up the vehicle; wherein the control link is a control parameter transmission link composed of at least the vehicle control cloud server module and the vehicle; The audio and video cloud server module is used to transmit the video control parameters actively sent by the mobile terminal to the vehicle via a video transmission link, so that the vehicle can shoot a target video clip according to the video control parameters; wherein the video transmission link is a multimedia data transmission link composed of at least the audio and video cloud server module and the vehicle; The audio and video cloud server module is used to receive the target video segment transmitted by the video transmission link, and transmit the target video segment to the mobile terminal for viewing.

9. A vehicle, characterized in that: include: a first receiving unit configured to receive a function activation command proactively issued by a mobile terminal bound to the vehicle and transmitted via a control link when the mobile terminal has a video viewing requirement, and to perform a wake-up operation according to the function activation command; wherein the control link is a control parameter transmission link composed of at least a vehicle control cloud server and the vehicle; a second receiving unit, configured to receive video control parameters proactively sent by the mobile terminal and transmitted via a video transmission link, and capture a target video segment according to the video control parameters; wherein the video transmission link is a multimedia data transmission link consisting of at least an audio and video cloud server and the vehicle; The sending unit is used to transmit the target video segment to the audio and video cloud server through the video transmission link, so that the audio and video cloud server forwards the target video segment to the mobile terminal for viewing.

10. A system for remotely monitoring video inside and outside a vehicle, characterized in that: The system includes: a mobile terminal, a cloud server and a vehicle; the cloud server includes an independent audio and video cloud server and a vehicle control cloud server; wherein, The mobile terminal is used to actively send function start commands and video control parameters when there is a need to view a video; The cloud server is configured to transmit the function activation command to the vehicle bound to the mobile terminal via a control link, and to transmit the video control parameters to the vehicle via a video transmission link; wherein the control link is a control parameter transmission link composed of at least the vehicle control cloud server and the vehicle; and the video transmission link is a multimedia data transmission link composed of at least the audio and video cloud server and the vehicle; The vehicle is configured to receive the function start command and perform a wake-up operation according to the function start command; and receive the video control parameter and shoot a target video clip according to the video control parameter; The cloud server is further configured to receive the target video segment transmitted by the video transmission link, and transmit the target video segment to the mobile terminal for viewing.

Images