Vehicle remote video viewing method and system and storage medium

Abstract

The invention relates to the field of remote control, and discloses a vehicle remote video viewing method and system and a storage medium, which are applied to an automatic driving controller, and the method comprises the following steps: when a vehicle is in a low power consumption state and a video viewing bus signal sent by vehicle-mounted communication equipment is received, waking up the automatic driving controller, the method comprises the steps of receiving a video viewing bus signal, determining each target camera corresponding to the video viewing bus signal, generating wake-up information, sending the wake-up information to each target camera so as to wake up each target camera to collect a single-path video signal, and keeping other modules connected with an automatic driving controller in a dormant state; and receiving the single-channel video signals acquired by each target camera, determining a combined video signal according to each single-channel video signal, and sending the combined video signal to the vehicle machine display screen so as to send the combined video signal to the target mobile terminal. Through the timely scheme of the invention, the effect of checking videos around the vehicle in a low-power-consumption state is realized, and the power consumption of the whole vehicle is reduced.

Description

Technical Field

[0001] The present invention relates to the field of remote control, and in particular to a vehicle remote video viewing method, system and storage medium. Background Art

[0002] With the evolution of existing vehicle-mounted products such as driving recorders, vehicle-mounted communication equipment, and vehicle-mounted entertainment systems, it is possible to remotely request real-time online video footage around the vehicle through mobile terminal devices, making it easier to visualize the vehicle's surroundings while it is driving or parked.

[0003] However, when the vehicle is parked and in a low-power state, requesting video images of the vehicle's surroundings through a mobile terminal device will cause the entire vehicle to wake up, resulting in unnecessary energy consumption.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a vehicle remote video viewing method, system and storage medium, which can achieve the effect of viewing the vehicle's surrounding video in a low-power state and reduce the power consumption of the entire vehicle.

[0006] An embodiment of the present invention provides a vehicle remote video viewing method, which is applied to an automatic driving controller. The method includes:

[0007] In response to the vehicle being in a low power consumption state and receiving a video viewing bus signal sent by an on-board communication device, waking up the automatic driving controller, determining each target camera corresponding to the video viewing bus signal, generating a wake-up message, sending the wake-up message to each target camera to wake up each target camera, collecting a single-channel video signal through each target camera, and keeping other modules connected to the automatic driving controller in a dormant state;

[0008] Receiving single-channel video signals collected by each target camera, determining a combined video signal according to each single-channel video signal, and sending the combined video signal to the vehicle display screen, so that the vehicle display screen sends the combined video signal to the target mobile terminal;

[0009] The video viewing bus signal is a bus signal determined by the vehicle-mounted communication device according to a video viewing request, and the video viewing request is a request signal sent by the target mobile terminal to the vehicle-mounted communication device.

[0010] An embodiment of the present invention provides a vehicle remote video viewing method, which is applied to a vehicle-mounted communication device. The method includes:

[0011] In response to the vehicle being in a low power consumption state, determining whether a video viewing request sent by a target mobile terminal is received based on a preset period;

[0012] In response to receiving the video viewing request, waking up the vehicle-mounted communication device, and determining a video viewing bus signal according to the video viewing request, and sending the video viewing bus signal to the automatic driving controller and the vehicle display screen to wake up the automatic driving controller and the vehicle display screen, and keep other hardware devices in a dormant state;

[0013] receiving a combined video signal sent by the automatic driving controller via the vehicle display screen, determining a video motion signal according to the combined video signal, and sending the video motion signal to the target mobile terminal;

[0014] Among them, the combined video signal is a video signal determined by the automatic driving controller based on a single-channel video signal collected by multiple awakened target cameras.

[0015] The embodiment of the present invention provides a vehicle remote video viewing system, characterized in that it includes: at least one front-view camera, at least one rear-view camera, at least four surround-view cameras, an automatic driving controller, a vehicle display screen, a vehicle-mounted communication device, and a target mobile terminal;

[0016] The front-view camera, the rear-view camera and the surround-view camera are connected to the automatic driving controller via a gigabit multimedia serial link, the automatic driving controller is connected to the vehicle display screen via Ethernet, the vehicle display screen and the automatic driving controller are respectively connected to the vehicle-mounted communication device via Ethernet, and the vehicle-mounted communication device is connected to the target mobile terminal via a mobile transmission network;

[0017] The vehicle-mounted communication device is used to perform the steps of remotely viewing the vehicle video in any embodiment;

[0018] The autonomous driving controller is used to execute the steps of remote vehicle video viewing described in any embodiment.

[0019] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, wherein the program or instruction enables a computer to execute the steps of the vehicle remote video viewing method described in any embodiment.

[0020] The embodiments of the present invention have the following technical effects:

[0021] In response to the vehicle being in a low-power state and receiving a video viewing bus signal sent by an on-board communication device, the automatic driving controller is awakened, and each target camera corresponding to the video viewing bus signal is determined, and a wake-up message is generated, and the wake-up message is sent to each target camera to wake up each target camera, and a single-channel video signal is collected through each target camera, and other modules connected to the automatic driving controller are kept in a dormant state. Then, the single-channel video signal collected by each target camera is received, and a combined video signal is determined according to each single-channel video signal, and the combined video signal is sent to the vehicle display screen, so that the vehicle display screen sends the combined video signal to the target mobile terminal, thereby achieving the effect of viewing the video around the vehicle in a low-power state, avoiding waking up other modules, and reducing the power consumption of the entire vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a flow chart of a vehicle remote video viewing method applied to an automatic driving controller provided by an embodiment of the present invention;

[0024] Figure 2 is a flow chart of a vehicle remote video viewing method applied to an in-vehicle communication device provided by an embodiment of the present invention;

[0025] Figure 3 is a structural schematic diagram of a vehicle remote video viewing system provided by an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of a wake-up part and a sleep part in an automatic driving controller provided by an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of an image processing flow of an automatic driving controller provided by an embodiment of the present invention;

[0028] Figure 6 is a schematic diagram of a preset video arrangement method provided by an embodiment of the present invention;

[0029] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0031] The vehicle remote video viewing method provided by the embodiment of the present invention is mainly applicable to the case where low-power remote viewing of the vehicle surrounding video is achieved when the vehicle is in a low-power state. The vehicle remote video viewing method provided by the embodiment of the present invention can be executed by an automatic driving controller or a vehicle-mounted communication device integrated in the vehicle.

[0032] Figure 1 is a flow chart of a vehicle remote video viewing method applied to an automatic driving controller provided by an embodiment of the present invention. Figure 1 , the vehicle remote video viewing method specifically includes:

[0033] S110, in response to the vehicle being in a low power consumption state and receiving a video viewing bus signal sent by an on-board communication device, wakes up the automatic driving controller, determines each target camera corresponding to the video viewing bus signal, generates a wake-up message, sends the wake-up message to each target camera to wake up each target camera, collects a single-channel video signal through each target camera, and keeps other modules connected to the automatic driving controller in a dormant state.

[0034] Among them, the autonomous driving controller (ADCU) is mainly used in intelligent driving scenarios and requires external environmental perception sensors, such as camera sensors, millimeter wave radars, laser radars, and IMU (Inertial Measurement Unit). Low power consumption is a working mode that can reduce energy consumption. The on-board communication equipment (Telematics Box, TBOX) integrates long-distance communication and information science and technology, and can provide services such as driving data collection, remote query and control. The video viewing bus signal is a bus signal determined by the on-board communication equipment according to the video viewing request, and the video viewing bus signal is a bus signal used to wake up the autonomous driving controller for video acquisition and transmission. The video viewing request is a request signal sent by the target mobile terminal to the on-board communication equipment, which is used to remotely request to view the video around the vehicle. The target mobile terminal is a mobile terminal that issues a video viewing request, which can be understood as a mobile terminal that can remotely control the vehicle. The target camera is the camera required for remote request to collect video, which can include a front-view camera, a rear-view camera, and a surround-view camera. The wake-up information is information used to wake up the target camera so that the target camera can work to collect video signals. The single-channel video signal is the video signal collected after the target camera is awakened by the wake-up information corresponding to the video viewing bus signal. Other modules are the modules connected to the automatic driving controller except the target cameras. The sleep state is the state corresponding to the working state, and the sleep mode is in the working state after being awakened.

[0035] Specifically, when the vehicle is in a low-power state, if the automatic driving controller receives a video viewing bus signal sent by the on-board communication device, it means that there is a request to remotely view the video around the vehicle in a low-power state. Therefore, it is necessary to wake up the automatic driving controller in order to call the sensor for collecting video under the automatic driving controller, that is, the target camera. Therefore, it is necessary to determine each target camera corresponding to the video viewing bus signal, generate a wake-up message, and send the wake-up message to each target camera to wake up each target camera. In this case, a single-channel video signal can be collected by each target camera, and other modules connected to the automatic driving controller can be kept in a dormant state to maximize the low-power state. That is, only each target camera is woken up to collect a single-channel video signal, and other modules that do not participate in the remote viewing of the video around the vehicle can remain in a dormant state.

[0036] Based on the above example, after waking up the automatic driving controller, in order to avoid waking up each target ultrasonic sensor, the following processing can be used, which can be:

[0037] Check the bus signal according to the video, determine each target ultrasonic sensor, and determine the target communication interface corresponding to each target ultrasonic sensor;

[0038] Based on the microcontroller of the autonomous driving controller, power is cut off to the target communication interface.

[0039] The target ultrasonic sensors are ultrasonic sensors that do not need to be awakened by the video viewing bus signal, and each target ultrasonic sensor is connected to the autonomous driving controller. The target communication interface is the communication interface between the autonomous driving controller and each target ultrasonic sensor, which can be DSI3 (3rd generation Distributed System Interface, a kind of automotive application bus interface). The autonomous driving controller includes a connected SOC (System onChip) and MCU (Microcontroller Unit), wherein the microcontroller is connected to each target ultrasonic sensor through the target communication interface.

[0040] Specifically, the bus signal is analyzed by video viewing to determine other modules that do not need to be awakened, from which the target ultrasonic sensors are determined, and further, the target communication interfaces corresponding to the target ultrasonic sensors are determined. The power supply of the target communication interface is cut off through the microcontroller of the autonomous driving controller to avoid awakening the target ultrasonic sensors from the hardware level.

[0041] It is understandable that, normally, the target ultrasonic sensors will be woken up when the autonomous driving controller is in operation. However, when the autonomous driving controller is woken up by the video viewing bus signal, the DSI3 power supply to each target ultrasonic sensor is cut off by the MCU, and the SOC only retains the minimum system necessary for remotely viewing the video around the vehicle, that is, only the video acquisition and transmission related services are retained in the ADCU.

[0042] Based on the above example, after waking up the automatic driving controller, in order to avoid waking up other modules, the following methods can be used to handle it, specifically:

[0043] View the bus signal according to the video, determine each other module, and determine the target sub-node corresponding to each other module;

[0044] For each target sub-node, close the network management sending end of the target sub-node.

[0045] Among them, the target subnode is the communication node between the autonomous driving controller and other modules, and the network management sending end is the port for network communication with other modules.

[0046] Specifically, the bus signal is analyzed in the video to determine the other modules that do not need to be awakened, and then the nodes through which the autonomous driving controller transmits signals to the other modules can be determined, that is, the target sub-nodes corresponding to the other modules. For each target sub-node, the network management sending end of the target sub-node needs to be turned off to avoid sending wake-up information to other modules so that other modules are awakened unnecessarily.

[0047] The above two methods are respectively ways to prevent other modules from being woken up from the hardware level and the software level, which can ensure that the vehicle is in a low power consumption state when viewing the video around the remote vehicle.

[0048] S120, receiving the single-channel video signals collected by each target camera, determining a combined video signal according to each single-channel video signal, and sending the combined video signal to the vehicle display screen, so that the vehicle display screen sends the combined video signal to the target mobile terminal.

[0049] The combined video signal is a video signal obtained by arranging the single-channel video signals. In this example, the vehicle display screen is used to transfer and transmit the combined video signal.

[0050] Specifically, after each target camera is awakened, it can collect a single-channel video signal and send it to the automatic driving controller so that the automatic driving controller can receive each single-channel video signal. Then, each single-channel video signal is arranged according to a preset arrangement to obtain a combined video signal. The combined video signal is sent to the vehicle display screen, so that the combined video signal is sent to the target mobile terminal through the vehicle display screen. At this time, the vehicle display screen is used as a transfer, and there is no need to process and display the combined video signal.

[0051] Based on the above example, the following methods can be used to receive the single-channel video signals collected by each target camera, determine the combined video signal based on each single-channel video signal, and send the combined video signal to the vehicle display screen:

[0052] Receive single-channel video signals collected by at least one front-view camera, at least one rear-view camera, and at least four surround-view cameras;

[0053] For each single-channel video signal, the single-channel video signal is processed by an image signal processor to obtain a process video signal, and the process video signal is compressed and encoded by an encoder to obtain an encoded video signal corresponding to the single-channel video signal;

[0054] Based on a preset video arrangement, the encoded video signals are combined to obtain a combined video signal;

[0055] Based on the real-time transmission protocol, the combined video signal is actively pushed to the car display screen.

[0056] Among them, the Image Signal Processing (ISP) can post-process a single-channel video signal to optimize the image quality. The process video signal is the signal output after the single-channel video signal is processed by the image signal processor. The encoder can encode the signal or data stream (usually for transmission, storage or encryption) or extract a coded stream. The encoded video signal is the output signal of the process video signal after the encoder performs video compression encoding processing. The video arrangement method is a method of queuing according to preset data, such as an arrangement method according to a preset size, a preset number of rows and a preset number of columns. The Real-Time Transport Protocol (RTP) is a network protocol for real-time transmission of audio and video data.

[0057] Specifically, the single-channel video signals collected by each front-view camera, each rear-view camera and each surround-view camera are received. Each single-channel video signal is processed separately. First, the image quality of the single-channel video signal is enhanced by an image signal processor to obtain a process video signal. Then, the process video signal is compressed and encoded by an encoder to obtain an encoded video signal corresponding to the single-channel video signal. After obtaining the encoded video signals of each channel, the encoded video signals are arranged and combined according to a preset video arrangement method to obtain a combined video signal. Furthermore, the combined video signal is actively pushed to the vehicle display screen through a real-time transmission protocol, so that the combined video signal can be sent to the target mobile terminal via the vehicle display screen, so that the user can view the video corresponding to the combined video signal on the target mobile terminal.

[0058] Based on the above example, if the target mobile terminal determines to stop viewing the video around the vehicle, the autonomous driving controller can be put back into a dormant state, which can be:

[0059] In response to receiving a stop viewing bus signal sent by the vehicle-mounted communication device, generating sleep information corresponding to each target camera based on the stop viewing bus signal;

[0060] Sending sleep information to each target camera to put each target camera into sleep state;

[0061] In response to each target camera entering a sleep state, the automatic driving controller is controlled to enter a sleep state.

[0062] The stop viewing bus signal is a bus signal for exiting the remote viewing of the video around the vehicle, and may be a bus signal generated by the vehicle-mounted communication device based on the stop viewing request, and the stop viewing request is a command request for exiting the viewing of the video around the vehicle triggered by the user based on the target mobile terminal. The sleep information is a command information for controlling each target camera to enter a sleep state.

[0063] Specifically, if a stop viewing bus signal is received from the vehicle-mounted communication device, it means that there is no need to continue collecting and transmitting video signals. Therefore, the autonomous driving controller and each target camera awakened by the autonomous driving controller can be put back into sleep mode. The stop viewing bus signal is analyzed, sleep information corresponding to each target camera is generated, and the sleep information is sent to each target camera, so that each target camera responds to the sleep information when it receives it and enters sleep mode. If all target cameras enter sleep mode, it means that all devices or modules under the autonomous driving controller are in sleep mode. Therefore, the autonomous driving controller can be controlled to enter sleep mode.

[0064] The present invention has the following technical effects: in response to the vehicle being in a low power consumption state and receiving a video viewing bus signal sent by an on-board communication device, the automatic driving controller is awakened, and each target camera corresponding to the video viewing bus signal is determined, and a wake-up message is generated, and the wake-up message is sent to each target camera to wake up each target camera, and a single-channel video signal is collected through each target camera, and other modules connected to the automatic driving controller are kept in a dormant state, and then, the single-channel video signal collected by each target camera is received, and a combined video signal is determined according to each single-channel video signal, and the combined video signal is sent to the vehicle display screen, so that the vehicle display screen sends the combined video signal to the target mobile terminal, thereby achieving the effect of viewing the video around the vehicle in a low power consumption state, avoiding waking up other modules, and reducing the power consumption of the entire vehicle.

[0065] Figure 2 is a flow chart of a vehicle remote video viewing method applied to a vehicle-mounted communication device provided by an embodiment of the present invention. Figure 2 , the vehicle remote video viewing method specifically includes:

[0066] S210: In response to the vehicle being in a low power consumption state, determining based on a preset period whether a video viewing request sent by a target mobile terminal is received.

[0067] The preset period is a period for the vehicle to establish a communication handshake mechanism by periodically polling with the target mobile terminal using the vehicle-mounted communication device in a low power consumption state, for example, it may be 1 second.

[0068] Specifically, if the vehicle is in a low power consumption state, the vehicle-mounted communication device can establish a communication handshake mechanism by polling with the target mobile terminal according to a preset period. At this time, when the user remotely views the video around the vehicle through the target mobile terminal, the target mobile terminal will issue a video viewing request, and the vehicle-mounted communication device can determine whether the video viewing request sent by the target mobile terminal is received, so as to further determine whether to perform subsequent low power consumption wake-up and video viewing transmission operations.

[0069] S220. In response to receiving a video viewing request, the vehicle-mounted communication device is awakened, and according to the video viewing request, a video viewing bus signal is determined, and the video viewing bus signal is sent to the automatic driving controller and the vehicle display screen to wake up the automatic driving controller and the vehicle display screen, and keep other hardware devices in a dormant state.

[0070] Among them, other hardware devices are hardware devices in the vehicle except for on-board communication equipment, automatic driving controller and vehicle display screen.

[0071] Specifically, if the vehicle-mounted communication device receives a video viewing request, it is necessary to first wake up the vehicle-mounted communication device itself, and it is necessary to wake up the autonomous driving controller and the vehicle display screen that are in communication with it through the vehicle-mounted communication device to obtain and transmit the video signal around the vehicle. Therefore, the video viewing request is converted into a video viewing bus signal, and the video viewing bus signal is sent to the autonomous driving controller and the vehicle display screen, so that the autonomous driving controller and the vehicle display screen are awakened when receiving the video viewing bus signal, while other hardware devices will not receive the video bus viewing signal and remain in a dormant state, so that the hardware devices called can be minimized.

[0072] S230, receiving a combined video signal sent by the automatic driving controller via the vehicle display screen, determining a video motion signal based on the combined video signal, and sending the video motion signal to a target mobile terminal.

[0073] The combined video signal is a video signal determined by the automatic driving controller based on a single-channel video signal collected by multiple awakened target cameras. The video motion signal is a motion signal corresponding to the combined video signal sent to the target mobile terminal.

[0074] Specifically, the automatic driving controller sends the combined video signal to the vehicle display screen, and the vehicle display screen forwards the received combined video signal to the vehicle communication device. When the vehicle communication device receives the combined video signal sent by the vehicle display screen, it converts the combined video signal into a video mobile signal for mobile signal transmission, and then sends the converted video mobile signal to the target mobile terminal via the mobile network, so that the user can view the video around the vehicle on the target mobile terminal.

[0075] Based on the above example, if the vehicle is powered on, it can also respond to video viewing requests. In this case, the video signal can be collected without maintaining a low power consumption state. Specifically, it can be:

[0076] In response to the vehicle being in a powered-on state and receiving a video viewing request, determining a video viewing bus signal according to the video viewing request, and sending the video viewing bus signal to the autonomous driving controller;

[0077] Receive the combined video signal sent by the automatic driving controller via the vehicle display screen, determine the video motion signal based on the combined video signal, and send the video motion signal to the target mobile terminal.

[0078] Specifically, if the vehicle receives a video viewing request sent by a target mobile terminal while it is powered on, it means that the vehicle does not need to maintain a low power consumption state. Therefore, the video viewing request can be directly converted into a video viewing bus signal, and the video viewing bus signal is sent to the automatic driving controller. The automatic driving controller receives and parses the video viewing bus signal, determines that each single-channel video signal collected by each target camera needs to be transmitted, and further, arranges each received single-channel video signal into a combined video signal after ISP processing and encoder processing, and transmits it to the vehicle display screen through RTP, and the vehicle display screen forwards the signal to the vehicle-mounted communication device. When the vehicle-mounted communication device receives the combined video signal, it converts the combined video signal into a video mobile signal for mobile transmission, and sends the video mobile signal to the target mobile terminal, so as to directly collect and transmit the video around the vehicle while the vehicle is powered on, without ensuring low power consumption processing.

[0079] Based on the above example, if the target mobile terminal determines to stop viewing the video around the vehicle, the vehicle-mounted communication device can be put back into a dormant state, specifically:

[0080] In response to receiving a stop viewing request, determining a stop viewing bus signal according to the stop viewing request, and sending the stop viewing bus signal to the automatic driving controller and the vehicle display screen to control the automatic driving controller and the vehicle display screen to enter a dormant state;

[0081] In response to the automatic driving controller and the vehicle display screen entering a sleep state, the vehicle-mounted communication device is controlled to enter a sleep state.

[0082] Specifically, if the vehicle-mounted communication device receives a stop viewing request, it means that there is no need to continue to collect and transmit video signals, so the vehicle-mounted communication device, the automatic driving controller, the vehicle display screen, and each target camera awakened by the automatic driving controller can be put into a dormant state again. Therefore, the stop viewing request is converted into a stop viewing bus signal for bus transmission, and the stop viewing bus signal is sent to the automatic driving controller and the vehicle display screen, so that the vehicle display screen directly enters a dormant state, and when the automatic driving controller receives the stop viewing bus signal, it controls each target camera to enter a dormant state, and can enter a dormant state itself. After determining that the automatic driving controller and the vehicle display screen have both entered a dormant state, the vehicle-mounted communication device can be controlled to enter a dormant state. If at least one of the automatic driving controller and the vehicle display screen does not enter a dormant state after a preset waiting time, the operation of sending the stop viewing bus signal to the automatic driving controller and the vehicle display screen is returned to control the automatic driving controller and the vehicle display screen to enter a dormant state again.

[0083] The present invention has the following technical effects: in response to the vehicle being in a low power consumption state, based on a preset period, it is determined whether a video viewing request sent by a target mobile terminal is received, and then, in response to receiving the video viewing request, the vehicle-mounted communication device is awakened, and according to the video viewing request, a video viewing bus signal is determined, and the video viewing bus signal is sent to the automatic driving controller and the vehicle display screen to wake up the automatic driving controller and the vehicle display screen, and keep other hardware devices in a dormant state, and further, a combined video signal sent by the automatic driving controller via the vehicle display screen is received, and a video movement signal is determined according to the combined video signal, and the video movement signal is sent to the target mobile terminal, thereby achieving the effect of viewing the video around the vehicle in a low power consumption state, avoiding waking up other hardware devices, and reducing the power consumption of the entire vehicle.

[0084] Figure 3 Schematic diagram of a vehicle remote video viewing system provided by an embodiment of the present invention. Figure 3 The vehicle remote video viewing system includes: at least one front-view camera 311, at least one rear-view camera 312, at least four surround-view cameras 313, an automatic driving controller 320, a vehicle display screen 330, a vehicle-mounted communication device 340 and a target mobile terminal 350.

[0085] Among them, the front-view camera 311, the rear-view camera 312 and the surround-view camera 313 are connected to the automatic driving controller 320 through Gigabit Multimedia Serial Links (GMSL), the automatic driving controller 320 is connected to the vehicle display screen 330 through Ethernet (Ethernet, Eth), the vehicle display screen 330 and the automatic driving controller 320 are respectively connected to the on-board communication device 340 through Ethernet, and the on-board communication device 340 is connected to the target mobile terminal 350 through a mobile transmission network (such as 4G, 5G, etc.); the on-board communication device 340 is used to execute the steps of vehicle remote video viewing applied to the on-board communication device 340 of any embodiment of the present invention; the automatic driving controller 320 is used to execute the steps of vehicle remote video viewing applied to the automatic driving controller 320 of any embodiment of the present invention.

[0086] When the vehicle is dormant (in low power consumption state), TBOX also enters low power consumption mode. The wireless signal of TBOX and the target mobile terminal (such as a mobile phone) is periodically polled to establish a communication handshake mechanism (for example, a period of 1 second). When the user remotely views the video around the vehicle through the target mobile terminal, TBOX converts the remote control information (video viewing request) sent by the target mobile terminal into a bus signal (video viewing bus signal). At this time, the mobile device control is in the on state, that is, Mobile RemoteControl = ON.

[0087] After TBOX polls and establishes wireless communication with the target mobile terminal, TBOX wakes itself up and sends a specific frame (video viewing bus signal, such as ID200) through Ethernet or CAN (Controller Area Network) or Flexray (a high-speed, deterministic, fault-tolerant bus technology for automobiles) to wake up the automatic driving controller and the vehicle display. Other nodes (other hardware devices) in the vehicle do not respond to this specific frame and cannot be started. It is understandable that the original vehicle used a global method, and all hardware devices would be woken up as long as they were woken up, which resulted in unnecessary power consumption.

[0088] After receiving the specific frame for wake-up from TBOX, the autopilot controller powers on and wakes up, shuts down the network management sending end of the subnode (the target subnode corresponding to each other module), and does not send out wake-up information to avoid waking up the corner radar, positioning module, etc. The other modules associated with the autopilot controller include: left front corner radar, right front corner radar, left rear corner radar, right rear corner radar, front radar, positioning module, etc.

[0089] In this case, the SOC of the autonomous driving controller receives the mode request signal (video viewing bus signal) Mobile Remote Control = ON from TBOX, and the MCU of the autonomous driving controller cuts off the DSI3 power supply to the USS (Ultrasonic Sensor System) and synchronizes it. The SOC only retains the minimum system necessary for remotely viewing the video around the vehicle, that is, only the video acquisition and transmission related services are retained in the ADCU. Figure 4 This is a schematic diagram of the wake-up and sleep parts in the autopilot controller. In this example, the AVM (Around view monitoring) and ACC (Adaptive Cruise Control) in the SOC marked in red, the CAN communication and DSI3 communication in the MCU are still in sleep mode, while the camera service and basic communication service in the SOC marked in green, the power management and Eth communication in the MCU are in wake-up mode. It can be seen that the green marked part is the minimized wake-up startup system in this example.

[0090] Figure 5 It is a schematic diagram of the image processing flow of the automatic driving controller, wherein at least one front-view camera, at least one rear-view camera and at least four surround-view cameras are used to collect single-channel video signals, and after ISP processing by the image signal processor, a process video signal is obtained, and then the video is compressed and encoded by the encoder to obtain each encoded video signal, which can be transmitted in a single-view mode; since remote viewing does not require real-time high fidelity, the transmission can also be combined and transmitted with RTP streams and small pictures, which can be arranged according to a preset video arrangement to obtain a combined video signal for transmission. Exemplarily, the RTP ADCU end actively pushes the stream, and the RTP Payload adopts H.264. In RTP, Payload refers to the data part carried in the RTP packet, FrameType (frame type) is IDR (Instantaneous Decoder Refresh, a specific frame type in H.264), BitRate (bit rate): 4Mb, FrameRate (frame rate): 30fps, image resolution: 1280*960; the preset video arrangement can be a 6-grid image, such as Figure 6 shown.

[0091] Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7As shown, the electronic device 400 includes one or more processors 401 and a memory 402. In this embodiment, the electronic device 400 may be a vehicle-mounted communication device or an automatic driving controller.

[0092] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0093] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the vehicle remote video viewing method of any embodiment of the present invention described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0094] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including early warning information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0095] Of course, to simplify, Figure 7 Only some of the components related to the present invention in the electronic device 400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 400 may also include any other appropriate components.

[0096] In addition to the above methods and devices, the embodiments of the present invention may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps of the vehicle remote video viewing method provided by any embodiment of the present invention.

[0097] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0098] In addition, the embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps of the vehicle remote video viewing method provided by any embodiment of the present invention.

[0099] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0101] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle remote video viewing method, characterized in that: Applied to autonomous driving controllers, including: In response to the vehicle being in a low power consumption state and receiving a video viewing bus signal sent by an on-board communication device, waking up the automatic driving controller, determining each target camera corresponding to the video viewing bus signal, generating a wake-up message, sending the wake-up message to each target camera to wake up each target camera, collecting a single-channel video signal through each target camera, and keeping other modules connected to the automatic driving controller in a dormant state; Receiving single-channel video signals collected by each target camera, determining a combined video signal according to each single-channel video signal, and sending the combined video signal to the vehicle display screen, so that the vehicle display screen sends the combined video signal to the target mobile terminal; The video viewing bus signal is a bus signal determined by the vehicle-mounted communication device according to a video viewing request, and the video viewing request is a request signal sent by the target mobile terminal to the vehicle-mounted communication device.

2. The method according to claim 1, characterized in that: After waking up the automatic driving controller, the method further includes: Viewing the bus signal according to the video, determining each target ultrasonic sensor, and determining the target communication interface corresponding to each target ultrasonic sensor; Based on the microcontroller of the autonomous driving controller, power supply of the target communication interface is cut off.

3. The method according to claim 1, characterized in that After waking up the automatic driving controller, the method further includes: According to the video viewing bus signal, determine each other module, and determine the target sub-node corresponding to each other module; For each target sub-node, the network management sending end of the target sub-node is closed.

4. The method according to claim 1, characterized in that: The receiving of the single-channel video signals collected by each target camera, determining a combined video signal according to each single-channel video signal, and sending the combined video signal to the vehicle display screen includes: Receive single-channel video signals collected by at least one front-view camera, at least one rear-view camera, and at least four surround-view cameras; For each single-channel video signal, the single-channel video signal is processed by an image signal processor to obtain a process video signal, and the process video signal is compressed and encoded by an encoder to obtain an encoded video signal corresponding to the single-channel video signal; Based on a preset video arrangement, the encoded video signals are combined to obtain a combined video signal; Based on the real-time transmission protocol, the combined video signal is actively pushed to the car display screen.

5. The method according to claim 1, characterized in that: Also includes: In response to receiving a stop viewing bus signal sent by the vehicle-mounted communication device, generating sleep information corresponding to each target camera based on the stop viewing bus signal; Sending the sleep information to each target camera to put each target camera into a sleep state; In response to each target camera entering a sleep state, the automatic driving controller is controlled to enter a sleep state.

6. A vehicle remote video viewing method, characterized in that: Applied to vehicle communication equipment, including: In response to the vehicle being in a low power consumption state, determining whether a video viewing request sent by a target mobile terminal is received based on a preset period; In response to receiving the video viewing request, waking up the vehicle-mounted communication device, and determining a video viewing bus signal according to the video viewing request, and sending the video viewing bus signal to the automatic driving controller and the vehicle display screen to wake up the automatic driving controller and the vehicle display screen, and keep other hardware devices in a dormant state; receiving a combined video signal sent by the automatic driving controller via the vehicle display screen, determining a video motion signal according to the combined video signal, and sending the video motion signal to the target mobile terminal; Among them, the combined video signal is a video signal determined by the automatic driving controller based on a single-channel video signal collected by multiple awakened target cameras.

7. The method according to claim 6, characterized in that Also includes: In response to the vehicle being in a powered-on state and receiving the video viewing request, determining a video viewing bus signal according to the video viewing request, and sending the video viewing bus signal to an automatic driving controller; Receive a combined video signal sent by the automatic driving controller via the vehicle display screen, determine a video motion signal based on the combined video signal, and send the video motion signal to the target mobile terminal.

8. The method according to claim 6, characterized in that Also includes: In response to receiving a stop viewing request, determining a stop viewing bus signal according to the stop viewing request, and sending the stop viewing bus signal to the automatic driving controller and the vehicle display screen to control the automatic driving controller and the vehicle display screen to enter a dormant state; In response to the automatic driving controller and the vehicle display screen entering a sleep state, the vehicle-mounted communication device is controlled to enter a sleep state.

9. A vehicle remote video viewing system, characterized in that: include: At least one front-view camera, at least one rear-view camera, at least four surround-view cameras, an autonomous driving controller, a vehicle display, an on-board communication device, and a target mobile terminal; The front-view camera, the rear-view camera and the surround-view camera are connected to the automatic driving controller via a gigabit multimedia serial link, the automatic driving controller is connected to the vehicle display screen via Ethernet, the vehicle display screen and the automatic driving controller are respectively connected to the vehicle-mounted communication device via Ethernet, and the vehicle-mounted communication device is connected to the target mobile terminal via a mobile transmission network; The vehicle-mounted communication device is used to perform the steps of vehicle remote video viewing as described in any one of claims 6 to 8; The automatic driving controller is used to execute the steps of vehicle remote video viewing as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the vehicle remote video viewing method according to any one of claims 1 to 5 or any one of claims 6 to 8.

Images