Method, vehicle-mounted terminal device and system for multi-terminal screen sharing in a vehicle

Abstract

The present disclosure relates to a method, in-vehicle terminal device and system for multi-terminal screen sharing in a vehicle. The method is applied to a master terminal device. The master terminal device, slave terminal devices in the vehicle and a domain controller are located in the same local area network. The method includes: after normal startup, starting a User Datagram Protocol (UDP) listening port and a socket service process of the master terminal device; establishing a long connection with a socket service process of the domain controller of the vehicle according to multicast messages listened on by the UDP listening port; rendering on a display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller; in response to a request initiated by a slave terminal device in the vehicle to connect to the socket service process of the master terminal device, establishing a long connection with the slave terminal device, and after successful connection, forwarding the vehicle perception and status data to the slave terminal device for independent rendering.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicles, and in particular, to a method for multi-terminal screen sharing in a vehicle, an in-vehicle terminal device, and a system. Background Art

[0002] With the continuous development of autonomous driving technology, the concept of an intelligent cockpit has emerged. How to enable passengers or drivers in the cockpit to more intuitively view the surrounding traffic elements perceived by the vehicle and how to perform data interaction between multiple screens has become particularly important.

[0003] In the process of implementing the technical concept of the present disclosure, the inventors found that there are at least the following technical problems in the related art: The current multi-screen content sharing in autonomous driving vehicles still adopts a relatively traditional screen sharing method, mirroring the screen image information of the master device to other screens in a live broadcast form, resulting in other screens being able to only watch and not interact. Summary of the Invention

[0004] To solve this technical problem or at least partially solve this technical problem, embodiments of the present disclosure provide a method for multi-terminal screen sharing in a vehicle, an in-vehicle terminal device, and a system.

[0005] In a first aspect, embodiments of the present disclosure provide a method for multi-terminal screen sharing in a vehicle. The method is applied to a master terminal device, and the master terminal device, a slave terminal device in the vehicle, and a domain controller are located in the same local area network. The method includes: after normal startup, starting a User Datagram Protocol (UDP) (UDP is a connectionless transport protocol) listening port and a socket service process of the master terminal device; establishing a long connection with the socket service process of the domain controller of the vehicle according to the multicast message listened to by the UDP listening port; rendering on the display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller; in response to a request from a slave terminal device in the vehicle to connect to the socket service process of the master terminal device, establishing a long connection with the slave terminal device, and after the connection is successful, forwarding the vehicle perception and status data to the slave terminal device for independent rendering.

[0006] According to an embodiment of the present disclosure, the master terminal device and the slave terminal device are divided into different logical groups, and a long connection is established with the socket service process of the domain controller of the vehicle according to the multicast message listened on by the UDP listening port, including: after the UPD listening port listens to a multicast message for the master terminal device, parsing the multicast message to obtain the target IP address of the domain controller, where the target IP address is used to transmit vehicle perception and status data; and establishing a long connection with the socket service process of the domain controller according to the target IP address.

[0007] According to an embodiment of the present disclosure, rendering on the display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller includes: performing obstacle detection according to the vehicle perception and status data to obtain a detection result of whether there is an obstacle and the corresponding obstacle type when there is an obstacle; when the detection result indicates the existence of an obstacle, determining the three-dimensional position information of the obstacle; and rendering the corresponding obstacle in the vehicle driving map that is real-time loaded on the display interface of the master terminal device according to the three-dimensional position information and the obstacle type.

[0008] According to an embodiment of the present disclosure, in the method applied to the master terminal device, the vehicle driving map is loaded in the following manner: obtaining the planned path of the vehicle; determining the map data to be displayed from the current driving position of the vehicle to the subsequent planned path; calculating a target map interval adapted to the current display parameters in the display interface of the master terminal device according to the map data, where the current display parameters are obtained by being pre-configured by the user or updated in real time; and real-time loading the data corresponding to the target map interval in the display interface to obtain the vehicle driving map.

[0009] According to another embodiment of the present disclosure, the above vehicle perception and status data includes at least one of the following: real-time traffic information, vehicle driving path information, vehicle status information, and rendering on the display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller includes: rendering at least one of the above real-time traffic information, vehicle driving path information, and vehicle status information on the display interface of the master terminal device.

[0010] According to an embodiment of the present disclosure, the master terminal device is an in-vehicle device on the driver side of the vehicle, and the slave terminal device is an in-vehicle device on the passenger side.

[0011] Second aspect, embodiments of the present disclosure provide a method for multi-terminal screen sharing in a vehicle. This method is applied to a slave terminal device, which is located in the same local area network as the master terminal device in the vehicle. The method includes: after normal startup, starting the Network Service Discovery (NSD) function to find the server in the local area network where it is located; when the server found based on the NSD function is the master terminal device in the state where the socket service process of the master terminal device is enabled, obtaining the IP address of the master terminal device; according to the IP address of the master terminal device, initiating a request to establish a long connection with the master terminal device; in the state where the socket service connection with the master terminal device is successful, receiving the vehicle perception and status data forwarded by the master terminal device; and rendering the vehicle perception and status data on the display interface of the slave terminal device according to the operating state of the slave terminal device.

[0012] According to an embodiment of the present disclosure, the vehicle perception and status data is a real-time updated data packet, and the data packet is adapted to the real-time driving position of the vehicle; rendering the vehicle perception and status data on the display interface of the slave terminal device according to the operating state of the slave terminal device includes: when it is detected that the currently loaded display interface of the slave terminal device is the target application, rendering the current data packet regarding the vehicle perception and status data on the currently loaded display interface of the slave terminal device; when it is detected that the currently loaded display interface of the slave terminal device is a non-target application, discarding the received current data packet regarding the vehicle perception and status data without rendering; when it is detected that the object loaded on the currently loaded display interface of the slave terminal device switches from a running non-target application to the target application, rendering the current data packet regarding the vehicle perception and status data on the currently loaded display interface of the slave terminal device.

[0013] According to an embodiment of the present disclosure, rendering the current data packet regarding the vehicle perception and status data on the currently loaded display interface of the slave terminal device includes: performing obstacle detection according to the current data packet regarding the perception and status data to obtain a detection result of whether there is an obstacle and the corresponding obstacle type when there is an obstacle; when the detection result indicates the existence of an obstacle, determining the three-dimensional position information of the obstacle; and rendering the corresponding obstacle in the vehicle driving map that is real-time loaded on the display interface of the slave terminal device according to the three-dimensional position information and the obstacle type.

[0014] According to an embodiment of the present disclosure, in the method applied to the slave terminal device, the vehicle driving map is loaded in the following manner: obtaining the planned path of the vehicle; determining the map data to be displayed from the current driving position of the vehicle to the subsequent planned path; calculating, according to the map data, a target map range adapted to the current display parameters in the display interface of the slave terminal device, where the current display parameters are obtained by being preconfigured by the user or updated in real time; and loading, in real time in the display interface, the data corresponding to the target map range to obtain the vehicle driving map.

[0015] According to another embodiment of the present disclosure, the above vehicle perception and status data includes at least one of the following: real-time traffic information, vehicle driving path information, and vehicle status information. Rendering the current data packet regarding the vehicle perception and status data in the current display interface of the slave terminal device includes: rendering at least one of the above real-time traffic information, vehicle driving path information, and vehicle status information in the display interface of the slave terminal device.

[0016] According to an embodiment of the present disclosure, the method applied to the slave terminal device further includes: sending, according to a preset task port in the server, a task creation request to the server for data interaction or data sharing with other slave terminal devices located in the same local area network as the slave terminal device; and performing data interaction or data sharing with the other slave terminal devices when receiving a feedback result of successful task creation fed back by the server.

[0017] In a third aspect, an embodiment of the present disclosure provides an in-vehicle terminal device. The in-vehicle terminal device is used for multi-terminal screen sharing in a vehicle. The in-vehicle terminal device serves as the master terminal device, and the master terminal device, the slave terminal devices in the vehicle, and the domain controller are located in the same local area network. The master terminal device includes: a listening start module, a communication service start module, a client identity connection module, a rendering module, a server identity connection module, and a data transmission module. The listening start module is used to start a UDP listening port after the normal startup of the master terminal device. The communication service start module is used to start the socket service process of the master terminal device after the normal startup of the master terminal device. The client identity connection module is used to establish a long connection with the socket service process of the vehicle's domain controller according to the multicast message listened by the UDP listening port. The rendering module is used to render, in the display interface of the master terminal device, the vehicle perception and status data received from the socket service process of the domain controller. The server identity connection module is used to establish a long connection with the slave terminal device in response to a request from the slave terminal device in the vehicle to connect to the socket service process of the master terminal device. The data transmission module is used to forward the vehicle perception and status data to the slave terminal device for independent rendering after a successful connection with the slave terminal device.

[0018] Fourthly, an embodiment of the present disclosure provides an in-vehicle terminal device. The in-vehicle terminal device is used for multi-terminal screen sharing in a vehicle. The in-vehicle terminal device serves as a slave terminal device, and the slave terminal device and the master terminal device in the vehicle are located in the same local area network. The slave terminal device includes: a service discovery module, a network information acquisition module, a client identity connection module, a data reception module, and a rendering module. The service discovery module is used to start the Network Service Discovery (NSD) function to discover the server in the local area network after the slave terminal device is normally started. The network information acquisition module is used to obtain the IP address of the master terminal device when the socket service process of the master terminal device is in an open state and the server found based on the NSD function is the master terminal device. The client identity connection module is used to initiate a request to establish a long connection with the master terminal device according to the IP address of the master terminal device. The data reception module is used to receive the vehicle perception and status data forwarded from the master terminal device when the socket service connection with the master terminal device is successful. The rendering module is used to render the vehicle perception and status data on the display interface of the slave terminal device according to the operating state of the slave terminal device.

[0019] Fifthly, an embodiment of the present disclosure provides a system for multi-terminal screen sharing in a vehicle. The system includes: a domain controller, a master terminal device, and a slave terminal device. The domain controller is used to send UDP multicast messages to the target port in the local area network of the vehicle and start the socket service process after normal startup, and is used to establish a long connection with the master terminal device based on the socket to transmit vehicle perception and status data. The master terminal device serves as a client of the vehicle's domain controller and at the same time serves as a server for the slave terminal device in the vehicle. It is used to establish long connections with the domain controller and the slave terminal device based on the socket respectively, and render the vehicle perception and status data received from the domain controller on the display interface of the master terminal device and forward the vehicle perception and status data to the slave terminal device respectively. The slave terminal device is used to establish a long connection with the master terminal device based on the socket and independently render the vehicle perception and status data received from the master terminal device on the display interface of the slave terminal device according to the operating state of the slave terminal device.

[0020] According to the embodiment of the present disclosure, the master terminal device and the slave terminal device are display devices built in the vehicle; or at least one of the master terminal device and the slave terminal device is a display device located in the vehicle and independent of the vehicle.

[0021] Some technical solutions provided by the embodiments of the present disclosure have some or all of the following advantages:

[0022] The master terminal device acts as both a client of the domain controller and a server of the slave terminal device in the vehicle. The master terminal device and the domain controller in the same local area network are connected to each other through UDP multicast and sockets. At the same time, the master terminal device and the slave terminal devices in the same local area network are connected to each other through sockets. The master terminal device can render the vehicle sensor and status data received from the domain controller by itself, and forward it to the slave terminal devices in the same local area network for independent rendering. Data can be shared between the master terminal device and the slave terminal devices, and can be displayed independently, thus achieving flexible screen sharing. In application scenarios, the slave terminal device (such as the passenger-side display device) and the master terminal device (such as the driver-side display device) can be rendered synchronously or asynchronously based on the same data, and the rendering process depends on the parameter settings of each terminal device itself. It has both sharing and flexibility, which helps to improve the user experience of the smart cockpit and can be applied to ordinary vehicles or autonomous driving vehicles.

[0023] Some technical solutions provided by the embodiments of the present disclosure have some or all of the following advantages:

[0024] Based on the master-slave forwarding relationship between the master terminal device and the slave terminal device and the master terminal device and the slave terminal device are located in different logical groups, it is conducive to the realization of differentiated functions. For example, vehicle perception and status data must be forwarded from the master terminal device to the slave terminal device and will not be distributed to the slave terminal device by the domain controller. It is conducive to the transfer control of data on the master terminal device side and can ensure the rendering on the master terminal device side first, which is conducive to intuitive driving perception display. For ordinary vehicles, it is helpful for the driving safety of the driver. For data interaction between slave terminal devices (for example, playing the same game together in the same local area network, players can be opponents or collaborators in the game) or data sharing (for example, transferring files, photos, videos, etc.), based on the differences in logical groupings, there is no need to transfer through the master terminal device. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1ASchematically shows the system architecture of a system for screen sharing among multiple terminals in a vehicle according to an embodiment of the present disclosure;

[0028] Figure 1B Schematically shows a schematic diagram of the interaction process among a domain controller, a master terminal device, and a slave terminal device according to an embodiment of the present disclosure;

[0029] Figure 2 Schematically shows a flowchart of a method for screen sharing among multiple terminals in a vehicle applied to a master terminal device according to an embodiment of the present disclosure;

[0030] Figure 3 Schematically shows a detailed implementation flowchart of step S220 according to an embodiment of the present disclosure;

[0031] Figure 4 Schematically shows a detailed implementation flowchart of step S230 according to an embodiment of the present disclosure;

[0032] Figure 5 Schematically shows a flowchart of a method for screen sharing among multiple terminals in a vehicle applied to a slave terminal device according to an embodiment of the present disclosure;

[0033] Figure 6 Schematically shows a detailed implementation flowchart of step S550 according to an embodiment of the present disclosure;

[0034] Figure 7 Schematically shows a flowchart of a method for screen sharing among multiple terminals in a vehicle applied to a slave terminal device according to another embodiment of the present disclosure;

[0035] Figure 8 Schematically shows a structural block diagram of an in-vehicle terminal device according to an embodiment of the present disclosure; and

[0036] Figure 9 Schematically shows a structural block diagram of an in-vehicle terminal device according to another embodiment of the present disclosure. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are some but not all of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0038] The first exemplary embodiment of the present disclosure provides a system for screen sharing among multiple terminals in a vehicle.

[0039] Figure 1A Schematically shows the system architecture of a system for screen sharing among multiple terminals in a vehicle according to an embodiment of the present disclosure; Figure 1B Schematically shows a schematic diagram of the interaction process among the domain controller, the master terminal device, and the slave terminal device according to an embodiment of the present disclosure.

[0040] Refer to Figure 1A As shown, the system 100 for screen sharing among multiple terminals in a vehicle provided by an embodiment of the present disclosure includes: a domain controller 110, a master terminal device 120, and a slave terminal device 130. The domain controller 110, the master terminal device 120, and the slave terminal device 130 are located in the same local area network.

[0041] Refer to Figure 1A and Figure 1B As shown, the domain controller 110 is used to send a UDP multicast message to a target port in the local area network of the vehicle and start a socket service process after normal startup, and is used to establish a long connection with the master terminal device based on the socket to transmit vehicle perception and status data.

[0042] The domain controller will start the in-vehicle communication module by itself when starting, for example, the in-vehicle communication module is a functional module in the vehicle computer system telematics.

[0043] The Internet protocol suite supports a connectionless transport protocol, which is called the User Datagram Protocol (UDP). UDP provides a method for applications to send encapsulated IP data packets without establishing a connection, that is, UDP is a connectionless transport protocol.

[0044] A socket is an abstraction of the endpoints for two-way communication between application processes on different hosts in a network. A socket is one end of the process communication on the network, providing a mechanism for application layer processes to exchange data using network protocols. In terms of its position, a socket is connected to the application process above and the network protocol stack below, and is the interface for the application program to communicate through the network protocol and the interface for the application program to interact with the network protocol stack.

[0045] The master terminal device 120 and the slave terminal device 130 are divided into different logical groups. The master terminal device 120 and the domain controller 110 are connected based on UDP multicast and Socket connection. The master terminal device 120 serves as the data receiving end of the vehicle perception and status data sent by the domain controller 110.

[0046] Refer to Figure 1A and Figure 1BAs shown, the main terminal device 120 serves as a client of the domain controller 110 of the vehicle and as a server of the slave terminal device 130 of the vehicle. It is used to establish long connections with the domain controller 110 and the slave terminal device 130 respectively based on sockets, and render the vehicle perception and status data received from the domain controller 110 on the display interface of the main terminal device 120, and forward the vehicle perception and status data to the slave terminal device 130.

[0047] Referring to Figure 1A and Figure 1B As shown, the slave terminal device 130 is used to establish a long connection with the main terminal device 120 based on a socket, and independently render the vehicle perception and status data received from the main terminal device 120 on the display interface of the slave terminal device 130 according to the operating state of the slave terminal device 130.

[0048] In Figure 1A two slave terminal devices 130 are taken as examples, and actually the number of slave terminal devices can be adjusted correspondingly according to the actual situation.

[0049] According to an embodiment of the present disclosure, the main terminal device and the slave terminal device are display devices built in the vehicle; or at least one of the main terminal device and the slave terminal device is a display device located inside the vehicle and independent of the vehicle.

[0050] In one embodiment, the main terminal device is an in-vehicle device on the driver's side (spatially located inside the vehicle). The in-vehicle device can be a display device built in the vehicle and within the driver's field of vision, such as an in-cockpit in-vehicle display screen; or it can be a terminal device used by the driver and installed with a map navigation application, such as a mobile phone, a tablet computer, a laptop computer, a smart watch, a smart bracelet with a display screen, etc. The main terminal device is used to implement the method for multi-terminal screen sharing inside the vehicle provided by the embodiments of the present disclosure. For example, the map navigation application installed in the main terminal device includes program instructions or functional modules for implementing the method for multi-terminal screen sharing inside the vehicle provided by the embodiments of the present disclosure.

[0051] In one embodiment, the slave terminal device is an in-vehicle device on the passenger side inside the vehicle (the spatial position is inside the vehicle). The in-vehicle device can be a display device built into the vehicle and within the passenger's field of vision. For example, it can be an in-vehicle display screen installed behind the driver's seat and facing the passengers for the rear-row users to view (the specific position is not limited and can be flexibly set inside the vehicle according to actual scenario needs); it can also be a terminal device used by the passenger and installed with a map navigation application, such as a mobile phone, a tablet computer, a laptop computer, a smart watch, a smart bracelet with a display screen, etc. The slave terminal device is used to implement the method for multi-terminal screen sharing inside the vehicle provided by the embodiments of the present disclosure. For example, the map navigation application installed in the slave terminal device includes program instructions or functional modules for implementing the method for multi-terminal screen sharing inside the vehicle provided by the embodiments of the present disclosure.

[0052] For example, in one embodiment, the domain controller starts the Socket-Server process. After establishing a long connection with the in-vehicle device on the driver's side (an example of the master terminal device), it transmits road condition perception data to the in-vehicle device on the driver's side. After receiving the road condition perception data, the in-vehicle device on the driver's side can render it on the current display interface as needed. Additionally, the in-vehicle device on the driver's side starts the Socket-Server process. After establishing a long connection with the in-vehicle device on the passenger's side (an example of the slave terminal device), it forwards the road condition perception data to the in-vehicle device on the passenger's side; after receiving the road condition perception data, the in-vehicle device on the passenger's side can independently render the road condition perception data according to its own operating state. The in-vehicle device on the driver's side and the in-vehicle device on the passenger's side independently render the road condition perception data according to their own needs, their own operating states, and their own display parameters, etc.

[0053] Based on the system provided by the embodiments of the present disclosure, the master terminal device serves as both a client of the domain controller and a server of the slave terminal devices inside the vehicle. The master terminal device and the domain controller located in the same local area network are communicatively connected based on UDP multicast and sockets. At the same time, the master terminal device and the slave terminal devices belonging to the same local area network are communicatively connected based on sockets. The master terminal device can not only render the vehicle sensing and status data received from the domain controller by itself but also forward it to the slave terminal devices in the same local area network for independent rendering. While realizing data sharing between the master terminal device and the slave terminal devices, it can also be independently displayed, achieving flexible screen sharing. In the application scenario, the slave terminal device (such as the passenger-side display device) and the master terminal device (such as the driver-side display device) can render synchronously or asynchronously based on the same data, and the rendering process depends on the parameter settings of each terminal device itself, combining sharing and flexibility, which helps to improve the user experience of the intelligent cockpit and can be applied to the fields of ordinary vehicles or autonomous driving vehicles.

[0054] The second exemplary embodiment of the present disclosure provides a method for screen sharing among multiple terminals in a vehicle. The method provided in this embodiment can be applied to the master terminal device 120 in the system 100 of FIG. 1. The master terminal device 120, the slave terminal devices 130 in the vehicle, and the domain controller 110 (which can also be described as an industrial control computer) are located in the same local area network.

[0055] For example, in one embodiment, a map navigation application is installed in the master terminal device 120, and the map navigation application includes program instructions or functional modules for implementing the method.

[0056] Figure 2 Schematically shows a flowchart of a method for screen sharing among multiple terminals in a vehicle applied to a master terminal device according to an embodiment of the present disclosure.

[0057] Refer to Figure 2 As shown, the method for screen sharing among multiple terminals in a vehicle applied to a master terminal device provided by an embodiment of the present disclosure includes the following steps: S210, S220, S230, and S240.

[0058] In step S210, after normal startup, start the User Datagram Protocol (UDP) listening port and the socket service process of the master terminal device.

[0059] After the execution entity of the method is normally started, start the UDP listening port and the socket service process. For example, it can be after the master terminal device is normally started or after the master terminal device normally starts the map navigation application (a type of self-developed application), start the UDP listening port and the socket service process. In this state, the master terminal device serves as the server for the slave terminal devices, with the corresponding identity being the socket server identity, and the corresponding client being the slave device.

[0060] In step S220, establish a long connection with the socket service process of the domain controller of the vehicle according to the multicast message listened to by the UDP listening port.

[0061] The master terminal device determines the target IP address of the domain controller according to the multicast message listened to by the UDP listening port, so as to establish a Socket connection with the domain controller based on the target IP address, and transmit vehicle perception and status data based on this connection channel. In this state, the master terminal device serves as the client of the domain controller, with the corresponding identity being the socket client identity, and the corresponding server being the domain controller.

[0062] In step S230, render on the display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller.

[0063] The vehicle perception and status data is a real-time updated data packet, which is adapted to the real-time driving position of the vehicle. After receiving the real-time updated data packet from the domain controller, the main terminal device renders the data packet on the current display interface according to the current display parameters.

[0064] In step S240, in response to a request from a slave terminal device in the vehicle to connect to the socket service process of the main terminal device, a long connection is established with the slave terminal device, and after the connection is successful, the vehicle perception and status data is forwarded to the slave terminal device for independent rendering.

[0065] Based on steps S210 - S240, the main terminal device and the domain controller located in the same local area network communicate and connect based on UDP multicast and sockets. At the same time, the main terminal device and the slave terminal devices belonging to the same local area network communicate and connect based on sockets. The main terminal device can render the vehicle sensing and status data received from the domain controller by itself, and can also forward it to the slave terminal devices within the same local area network for independent rendering. While realizing data sharing between the main terminal device and the slave terminal devices, it can also be independently displayed, achieving flexible screen sharing. In the application scenario, the slave terminal device (such as the passenger-side display device) and the main terminal device (such as the driver-side display device) can render synchronously or asynchronously based on the same data, and the rendering process depends on the parameter settings of each terminal device itself, which has both sharing and flexibility, helps to improve the user experience of the intelligent cockpit, and can be applied to the field of ordinary vehicles or autonomous driving vehicles.

[0066] Figure 3 Schematically shows a detailed implementation flowchart of step S220 of the embodiments of the present disclosure.

[0067] According to an embodiment of the present disclosure, the main terminal device and the slave terminal device are divided into different logical groups. Refer to Figure 3 As shown, in step S220, according to the multicast message listened to by the UDP listening port, a long connection is established with the socket service process of the domain controller of the vehicle, including the following steps: S310 and S320.

[0068] In step S310, when the UPD listening port listens to a multicast message for the main terminal device, the multicast message is parsed to obtain the target IP address of the domain controller, and the target IP address is used to transmit the vehicle perception and status data.

[0069] The multicast message is sent by the domain controller 110 to the primary terminal device 110 within the same local area network and is not directed to the secondary terminal devices. The multicast message carries a custom protocol, which refers to the message content data structure. The content sent by the domain controller is a marker field, which is an IP address field used by the domain controller to transmit vehicle perception and status data. When the primary terminal device (such as the in-vehicle device on the driver side) receives the directed multicast message and parses it to obtain this marker field, a long connection request can be established.

[0070] In step S320, a long connection is established with the socket service process of the domain controller according to the target IP address.

[0071] Based on steps S310 - S320, a socket connection between the domain controller and the primary terminal device can be achieved to transmit vehicle sensing and status data.

[0072] Based on the master-slave forwarding relationship between the primary terminal device and the secondary terminal devices and the fact that the primary terminal device and the secondary terminal devices are in different logical groups, it is beneficial to the realization of differentiated functions. For example, for vehicle perception and status data, it must be forwarded from the primary terminal device to the secondary terminal devices and not directly distributed by the domain controller to the secondary terminal devices, which is conducive to data transfer control on the primary terminal device side and can ensure rendering on the primary terminal device side first, facilitating intuitive driving perception display and contributing to driving safety on the driver side for ordinary vehicles.

[0073] Figure 4 Schematically shows the detailed implementation flowchart of step S230 of the embodiment of the present disclosure.

[0074] According to an embodiment of the present disclosure, with reference to Figure 4 As shown, in this step S230, according to the vehicle perception and status data received from the socket service process of the domain controller, rendering is performed on the display interface of the primary terminal device, including the following steps: S410, S420, and S430.

[0075] In step S410, obstacle detection is performed according to the vehicle perception and status data to obtain the detection result of whether there is an obstacle and the corresponding obstacle type when there is an obstacle.

[0076] For example, vehicle perception and status data includes, for example, road condition images captured by on-vehicle camera devices. To detect obstacles in the road condition images, image segmentation can be performed first by invoking the image processing service of OpenCV (Open Source Computer Vision, a programming function library mainly for real-time computer vision). The segmented image regions are input into an obstacle detection model via TensorFlow (an open-source machine learning platform, a platform for transmitting complex data structures to artificial intelligence neural networks for analysis and processing) for recognition to obtain the detection results. The obstacle detection model is obtained by training with a large amount of road condition calibration data.

[0077] In step S420, when the detection result indicates the existence of an obstacle, determine the three-dimensional position information of the obstacle.

[0078] For example, the three-dimensional position information of the obstacle can be determined in the following way: According to the vehicle perception and status data and the parameters of the camera device, determine the relative position information of the obstacle in the world coordinate system where the vehicle is located; According to the real-time position information of the vehicle and the relative position information, determine the three-dimensional position information of the obstacle.

[0079] In step S430, according to the three-dimensional position information and the type of the obstacle, render the corresponding obstacle in the vehicle driving map that is real-time loaded on the display interface of the main terminal device.

[0080] In an embodiment, when it is detected that the currently loaded display interface of the main terminal device is a target application (for example, the target application is a map navigation application), render the current data packet regarding the vehicle perception and status data on the current display interface of the main terminal device; when it is detected that the currently loaded display interface of the main terminal device is a non-target application, discard the received current data packet regarding the vehicle perception and status data without rendering; when it is detected that the object loaded on the current display interface of the main terminal device is switched from a running non-target application to a target application, render the current data packet regarding the vehicle perception and status data on the current display interface of the main terminal device.

[0081] According to an embodiment of the present disclosure, in the method applied to the main terminal device, the vehicle driving map is loaded in the following way:

[0082] Obtain the planned path of the vehicle;

[0083] Determine the map data to be displayed from the current driving position of the vehicle to the subsequent planned path;

[0084] Calculate a target map interval adapted to the current display parameters in the display interface of the main terminal device according to the map data, where the current display parameters are obtained by being pre-configured or updated in real time by the user; and

[0085] Load the data corresponding to the target map section in real time on the display interface to obtain the vehicle driving map.

[0086] Based on the above loading method, it is possible to dynamically load the vehicle driving map during the vehicle's driving process, and be able to change the target map section corresponding to the displayed vehicle driving map according to the currently displayed parameters set or updated by the user, which helps to meet the personalized needs of users and improve the flexibility of the display of the vehicle driving map.

[0087] For autonomous vehicles or ordinary vehicles, the planned path of the vehicle can be obtained from a cloud server that provides navigation services for the vehicle or a navigation application built into the vehicle.

[0088] Among the planned road sections from the starting point to the ending point of the planned path, as the vehicle drives in real time, determine the map data to be displayed from the vehicle's current driving position to the subsequent planned path. For example, the map data is the map data to be displayed within the range from the current driving position to the ending point of the planned path; or the map data within multiple different distance intervals starting from the current driving position, such as the map data within 5 meters from the current driving position, the map data within 20 meters from the current driving position, the map data within 200 meters from the current driving position, the map data within 400 meters from the current driving position, the map data within 1000 meters from the current driving position, and so on.

[0089] For example, if the map data adapted to the currently displayed parameters in the display interface of the in-vehicle device is the map data within 200 meters from the current driving position, then the range from the current position to within 200 meters in the planned path is the target map section. In other embodiments, if the currently displayed parameters set by the user are different, then the target map section loaded in real time on the display interface of the in-vehicle unit is also different. For example, the range from the current position to within 20 meters in the planned path is the target map section. For example, the currently displayed parameters include parameters such as the length, width, height, and pixel density of the display screen of the in-vehicle device, as well as the viewing height set by the current user; the rendering range of the map is dynamically adjusted by combining the above parameter information.

[0090] According to another embodiment of the present disclosure, the above vehicle perception and status data includes at least one of the following: real-time traffic information, vehicle driving path information, vehicle status information. According to the vehicle perception and status data received from the socket service process of the domain controller, render on the display interface of the main terminal device, including: rendering at least one of the above real-time traffic information, vehicle driving path information, or vehicle status information on the display interface of the main terminal device.

[0091] For example, the real-time traffic information may be the information collected by vehicle sensing devices, such as the traffic information captured by vehicle cameras. The vehicle driving path information is the vehicle planned path information, and the vehicle state information includes the vehicle information sensed by in-vehicle sensors, such as the vehicle driving speed, vehicle fuel consumption, vehicle turning state, etc.

[0092] The third exemplary embodiment of the present disclosure provides a method for multi-terminal screen sharing in a vehicle. This method is applied to each slave terminal device 130 in the system 100 shown in FIG. 1, and the slave terminal device 130 and the master terminal device 110 in the vehicle are located in the same local area network.

[0093] For example, in one embodiment, a map navigation application is installed in the slave terminal device 130, and the map navigation application includes program instructions or functional modules for implementing this method.

[0094] Figure 5 Schematically shows a flowchart of a method for multi-terminal screen sharing in a vehicle applied to a slave terminal device according to an embodiment of the present disclosure.

[0095] Refer to Figure 5 As shown, the method for multi-terminal screen sharing in a vehicle applied to a slave terminal device provided by the embodiment of the present disclosure includes the following steps: S510, S520, S530, S540, and S550.

[0096] In step S510, after normal startup, start the Network Service Discovery (NSD) function to find the server in the local area network where it is located.

[0097] After the slave terminal device is normally started, start the NSD function to find the server in the local area network where it is located.

[0098] In step S520, when the socket service process of the master terminal device is in an open state and the server found based on the NSD function is the master terminal device, obtain the IP address of the master terminal device.

[0099] Since the master terminal device has started the socket service process, and the master terminal device, slave terminal device and domain controller in the same local area network have different functional attributes, for example, by setting the master terminal device and the slave terminal device in different logical groups, the domain controller can transmit pre-connection information to the master terminal device rather than the slave terminal device in the form of UDP multicast; similarly, when the slave terminal device searches for the server based on the NSD function, since a master-slave relationship is set with the master terminal device, that is, data is received through the master terminal device, it cannot bypass the domain controller to obtain data. When multiple servers are found based on the NSD function, the IP address of the master terminal device is obtained to establish a connection with the master terminal device.

[0100] In step S530, a request to establish a long connection is initiated with the main terminal device according to the IP address of the main terminal device.

[0101] In step S540, when the socket service connection with the main terminal device is successful, the vehicle perception and status data forwarded from the main terminal device is received.

[0102] In step S550, the vehicle perception and status data are rendered on a display interface of the slave terminal device according to the operating status of the slave terminal device.

[0103] Based on steps S510 to S550, the slave terminal device located in the same local area network in the vehicle establishes a long socket connection with the master terminal device, receives the vehicle perception and status data transferred from the master terminal device, and independently renders the vehicle perception and status data according to the operating status of the slave terminal device itself, without interfering with the current display of the slave terminal device and being able to adapt to the user's need to view the perception data at any time, thereby realizing flexible screen sharing between the master terminal device and the slave terminal device. In the application scenario, the slave terminal device (such as the passenger-side display device) and the master terminal device (such as the driver-side display device) can be rendered synchronously or asynchronously based on the same data, and the rendering process depends on the parameter settings of each terminal device itself, which is both shareable and flexible, helps to improve the user experience of the smart cockpit, and can be applied to ordinary vehicles or autonomous driving vehicles.

[0104] Figure 6 The detailed implementation flow chart of step S550 of the embodiment of the present disclosure is schematically shown.

[0105] According to an embodiment of the present disclosure, the vehicle perception and status data is a data packet updated in real time, and the data packet is adapted to the real-time driving position of the vehicle.

[0106] Reference Figure 6As shown, in step S550, according to the operating state of the slave terminal device, the vehicle perception and status data are rendered on the display interface of the slave terminal device, including at least one of the following steps: S610, S620 or S630, corresponding to the three scenarios of examples (a), (b) and (c) respectively.

[0107] In step S610, when it is detected that the currently loaded application on the display interface of the slave terminal device is the target application, the current data packet regarding the vehicle perception and status data is rendered on the current display interface of the slave terminal device.

[0108] In scenario (a), it can be that the slave terminal device has launched the target application (such as a map navigation application) and the currently loaded application on the display interface is exactly this target application. At the same time, it is not limited whether other applications are running in the background on the slave terminal device.

[0109] In step S620, when it is detected that the currently loaded application on the display interface of the slave terminal device is a non-target application, the currently received data packet regarding the vehicle perception and status data is discarded without rendering.

[0110] In scenario (b), if the currently loaded application on the display interface of the slave terminal device is a non-target application, even if the slave terminal device is running the target application in the background, the current data packet will not be rendered, which can not only save the resource consumption of running in the background but also does not affect the presentation of the currently displayed application.

[0111] In step S630, when it is detected that the object loaded on the current display interface of the slave terminal device is switched from a running non-target application to a target application, the current data packet regarding the vehicle perception and status data is rendered on the current display interface of the slave terminal device.

[0112] In scenario (c), the slave terminal device can run multiple applications simultaneously, including the target application and non-target applications. At a certain moment, the object loaded on the display interface can be a non-target application. When the user switches the running target application to the display interface for loading at the next moment, the slave terminal device will obtain the real-time data packet corresponding to that moment (the moment of loading and displaying the target application) and perform rendering.

[0113] Based on steps S610 - S630, the display priority of the current object loaded on the display interface of the slave terminal device is set to the highest. When the object loaded on the current display interface is the target application (the state of being switched from a non-target application to a target application or the state of currently displaying the target application), such as a map navigation application, the real-time received vehicle perception and status data will be rendered, without interfering with the current display of the slave terminal device and being able to meet the user's need to view the perception data at any time.

[0114] According to an embodiment of the present disclosure, rendering a current data packet regarding the vehicle perception and status data on the current display interface of the slave terminal device includes: performing obstacle detection according to the current data packet regarding the perception and status data to obtain a detection result of whether there is an obstacle and the corresponding obstacle type when there is an obstacle; when the detection result indicates the existence of an obstacle, determining the three-dimensional position information of the obstacle; and rendering the corresponding obstacle in the vehicle driving map that is real-time loaded on the display interface of the slave terminal device according to the three-dimensional position information and the obstacle type.

[0115] According to an embodiment of the present disclosure, in the method applied to the slave terminal device, the vehicle driving map is loaded in the following manner: obtaining the planned path of the vehicle; determining the map data to be displayed from the current driving position of the vehicle to the subsequent planned path; calculating a target map interval adapted to the current display parameters in the display interface of the slave terminal device, where the current display parameters are obtained by being pre-configured or updated in real time by the user; and real-time loading the data corresponding to the target map interval in the display interface to obtain the vehicle driving map.

[0116] According to another embodiment of the present disclosure, the above vehicle perception and status data includes at least one of the following: real-time traffic condition information, vehicle driving path information, vehicle status information. Rendering a current data packet regarding the vehicle perception and status data on the current display interface of the slave terminal device includes: rendering at least one of the above real-time traffic condition information, vehicle driving path information, or vehicle status information on the display interface of the slave terminal device.

[0117] For example, the real-time traffic condition information may be information collected by vehicle sensing devices, such as traffic condition information obtained by shooting with a vehicle camera. The vehicle driving path information is the vehicle planned path information. The vehicle status information includes vehicle information sensed by in-vehicle sensors, such as vehicle driving speed, vehicle fuel consumption, vehicle turning status, etc.

[0118] In this embodiment, the logic of data rendering by the slave terminal device is similar to the logic of data rendering by the master terminal device in the second embodiment. The rendering process applicable to the master terminal device in the second embodiment can also be applied to the slave terminal device. The content of the second embodiment can be incorporated by reference into this embodiment and will not be elaborated here.

[0119] Figure 7 Schematically shows a flowchart of a method for screen sharing among multiple terminals in a vehicle applied to a slave terminal device according to another embodiment of the present disclosure.

[0120] According to an embodiment of the present disclosure, the method applied to the slave terminal device further includes the following steps in addition to the steps S510 to S550: S710 and S720. Refer to Figure 7 As shown, for simplicity of illustration, only the steps S710 and S720 are schematically shown in Figure 7 .

[0121] In step S710, according to the preset task port in the server, a task creation request for data interaction or data sharing with other slave terminal devices located in the same local area network as the slave terminal device is sent to the server.

[0122] In step S720, after receiving the feedback result of successful task creation from the server, data interaction or data sharing is performed with the other slave terminal devices.

[0123] The steps S710 to S720 and the foregoing steps S510 to S550 can be executed independently of each other.

[0124] Scenarios of data interaction include, for example, but are not limited to: multiple users of slave terminals under the same local area network play the same game together, and the players can be opponents or collaborators in the game.

[0125] Scenarios of data sharing include, for example, but are not limited to: multiple slave terminal devices under the same local area network transfer files to each other, such as photos, videos, etc.

[0126] The interaction information flow between the second embodiment and the third embodiment can be understood with reference to Figure 1A and Figure 1B the exemplary system and the data interaction process.

[0127] A fourth exemplary embodiment of the present disclosure provides a vehicle-mounted terminal device. The vehicle-mounted terminal device is used for multi-terminal screen sharing in a vehicle. The vehicle-mounted terminal device serves as the master terminal device, and the master terminal device, the slave terminal devices in the vehicle, and the domain controller are located in the same local area network. The master terminal device serves as both the client of the vehicle's domain controller and the server of the vehicle's slave terminal devices.

[0128] Figure 8 The structural block diagram of the vehicle-mounted terminal device according to an embodiment of the present disclosure is schematically shown.

[0129] Refer to Figure 8 As shown, the vehicle-mounted terminal device 800 provided by the embodiment of the present disclosure includes: a listening start module 801, a communication service start module 802, a client identity connection module 803, a rendering module 804, a server identity connection module 805, and a data transmission module 806.

[0130] The monitoring startup module 801 is used to start a UDP monitoring port after the main terminal device is normally started.

[0131] The communication service startup module 802 is used to start the socket service process of the main terminal device after the main terminal device is normally started.

[0132] The client identity connection module 803 is used to establish a long connection with the socket service process of the domain controller of the vehicle according to the multicast message monitored by the UDP monitoring port.

[0133] The rendering module 804 is used to render on the display interface of the main terminal device according to the vehicle perception and status data received from the socket service process of the domain controller.

[0134] The server identity connection module 805 is used to establish a long connection with the slave terminal device in response to a request from the slave terminal device in the vehicle to connect to the socket service process of the main terminal device.

[0135] The data transmission module 806 is used to forward the vehicle perception and status data to the slave terminal device for independent rendering after successfully connecting to the slave terminal device.

[0136] The fifth exemplary embodiment of the present disclosure provides an in-vehicle terminal device. The in-vehicle terminal device is used for multi-terminal screen sharing in a vehicle. The in-vehicle terminal device serves as a slave terminal device, and the slave terminal device and the main terminal device in the vehicle are located in the same local area network.

[0137] Figure 9 Schematically shows a structural block diagram of an in-vehicle terminal device according to another embodiment of the present disclosure.

[0138] Refer to Figure 9 As shown, the in-vehicle terminal device 900 provided by the embodiment of the present disclosure includes: a service discovery module 901, a network information acquisition module 902, a client identity connection module 903, a data reception module 904, and a rendering module 905.

[0139] The service discovery module 901 is used to start the Network Service Discovery (NSD) function to discover the server in the local area network after the slave terminal device is normally started.

[0140] The network information acquisition module 902 is used to obtain the IP address of the main terminal device when the server found based on the NSD function is the main terminal device in the state where the socket service process of the main terminal device is enabled.

[0141] The client identity connection module 903 is used to initiate a request to establish a long connection with the main terminal device according to the IP address of the main terminal device.

[0142] The data receiving module 904 is configured to receive the vehicle perception and status data forwarded from the master terminal device in a state where the socket service connection with the master terminal device is successful.

[0143] The rendering module 905 is configured to render the vehicle perception and status data on the display interface of the slave terminal device according to the operating state of the slave terminal device.

[0144] In some embodiments, the in-vehicle terminal device 900 may further include: an interaction module.

[0145] The interaction module is configured to initiate a task creation request for data interaction or data sharing with other slave terminal devices located in the same local area network as the slave terminal device to the server according to a preset task port in the server; and perform data interaction or data sharing with the other slave terminal devices after receiving a feedback result of successful task creation fed back by the server.

[0146] Any number of the functional modules in the in-vehicle terminal device 800 or 900 may be combined and implemented in one module, or any one of the modules may be split into multiple modules. Or, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules in the in-vehicle terminal device 800 or 900 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or may be implemented by any other reasonable means such as integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware or in an appropriate combination of any several of them. Or, at least one of the functional modules in the in-vehicle terminal device 800 or 900 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions may be executed.

[0147] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0148] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for multi-terminal screen sharing in a vehicle, characterized in that, Applied to a master terminal device, the master terminal device, the slave terminal device in the vehicle and the domain controller are located in the same local area network, the method comprises: After normal startup, start the UDP listening port and the socket service process of the main terminal device; Establishing a long connection with the socket service process of the domain controller of the vehicle according to the multicast message monitored by the UDP monitoring port; the multicast message is sent by the domain controller to the master terminal device in the same local area network, not to the slave terminal device; Rendering on a display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller; In response to a request initiated by a slave terminal device in the vehicle to connect to the socket service process of the master terminal device, establish a persistent connection with the slave terminal device, and after the connection is successful, forward the vehicle perception and status data to the slave terminal device for independent rendering; The master terminal device and the slave terminal device are divided into different logical groups, and the vehicle perception and status data are forwarded from the master terminal device to the slave terminal device instead of being distributed to the slave terminal device by the domain controller.

2. The method according to claim 1, characterized in that, According to the multicast message monitored by the UDP monitoring port, a long connection is established with the socket service process of the domain controller of the vehicle, including: When the UPD listening port listens to a multicast message for the master terminal device, the multicast message is parsed to obtain a target IP address of the domain controller, where the target IP address is used to transmit vehicle perception and status data; According to the target IP address, a long connection is established with the socket service process of the domain controller.

3. The method according to claim 1, characterized in that, Rendering on a display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller includes: Perform obstacle detection based on the vehicle perception and status data to obtain a detection result of whether an obstacle exists and the corresponding obstacle type when an obstacle exists; When the detection result indicates that an obstacle exists, determining three-dimensional position information of the obstacle; and According to the three-dimensional position information and the obstacle type, the corresponding obstacle is rendered in the vehicle driving map loaded in real time on the display interface of the main terminal device.

4. The method according to claim 3, characterized in that, The vehicle driving map is loaded in the following way: Obtaining a planned path of the vehicle; Determining map data to be displayed from the current driving position of the vehicle to a subsequent planned path; Calculating a target map interval adapted to current display parameters in a display interface of the master terminal device according to the map data, wherein the current display parameters are pre-configured by a user or updated in real time; as well as The data corresponding to the target map section is loaded in real time on the display interface to obtain the vehicle driving map.

5. The method according to any one of claims 1-4, characterized in that, The master terminal device is a vehicle-mounted device on the driver's side of the vehicle, and the slave terminal device is a vehicle-mounted device on the passenger's side.

6. A method for multi-terminal screen sharing in a vehicle, characterized in that, Applied to a slave terminal device, the slave terminal device is located in the same local area network as the master terminal device and the domain controller in the vehicle, the method comprises: After normal startup, start the network service discovery NSD function to find the server in the local area network; When the socket service process of the master terminal device is turned on, when the server found by the NSD function is the master terminal device, obtaining the IP address of the master terminal device; Initiate a request to establish a persistent connection with the main terminal device according to the IP address of the main terminal device; In a state where the socket service connection with the main terminal device is successful, receiving the vehicle perception and status data forwarded from the main terminal device; and Rendering the vehicle perception and status data on a display interface of the slave terminal device according to the operating status of the slave terminal device; Among them, the domain controller of the vehicle sends a multicast message to establish a long connection between the socket service process of the domain controller and the master terminal device in the vehicle; the multicast message is sent by the domain controller to the master terminal device in the same local area network, and is not directed to the slave terminal device; the domain controller sends vehicle perception and status data based on the socket service process; the master terminal device and the slave terminal device are divided into different logical groups, and the vehicle perception and status data are forwarded by the master terminal device to the slave terminal device instead of being distributed to the slave terminal device by the domain controller.

7. The method according to claim 6, characterized in that, The vehicle perception and status data is a data packet updated in real time, and the data packet is adapted to the real-time driving position of the vehicle; Rendering the vehicle perception and status data on a display interface of the slave terminal device according to the operating status of the slave terminal device includes: When it is detected that the current display interface of the slave terminal device is loaded with the target application, rendering the current data packet about the vehicle perception and status data on the current display interface of the slave terminal device; When it is detected that the current display interface of the slave terminal device is loaded with a non-target application, the current data packet received about the vehicle perception and status data is discarded without rendering; When it is detected that the object loaded on the current display interface of the slave terminal device is switched from a running non-target application to a target application, the current data packet about the vehicle perception and status data is rendered on the current display interface of the slave terminal device.

8. The method according to claim 7, characterized in that, Rendering a current data packet about the vehicle perception and status data on a current display interface of the slave terminal device includes: Performing obstacle detection according to the current data packet of the perception and status data to obtain a detection result of whether an obstacle exists and a corresponding obstacle type when an obstacle exists; When the detection result indicates that an obstacle exists, determining three-dimensional position information of the obstacle; and According to the three-dimensional position information and the obstacle type, the corresponding obstacle is rendered in the vehicle driving map loaded in real time on the display interface of the slave terminal device.

9. The method according to claim 8, characterized in that, The vehicle driving map is loaded in the following way: Obtaining a planned path of the vehicle; Determining map data to be displayed from the current driving position of the vehicle to a subsequent planned path; Calculating a target map interval adapted to current display parameters in a display interface of the slave terminal device according to the map data, wherein the current display parameters are pre-configured by a user or updated in real time; as well as The data corresponding to the target map section is loaded in real time on the display interface to obtain the vehicle driving map.

10. The method according to claim 6, characterized in that, Also includes: Initiate a task creation request to the server for data interaction or data sharing with other slave terminal devices located in the same local area network as the slave terminal device according to a preset task port in the server; After receiving the feedback result of successful task creation from the server, data interaction or data sharing is performed with the other slave terminal devices.

11. A vehicle-mounted terminal device, characterized in that, For performing multi-terminal screen sharing in a vehicle, the vehicle-mounted terminal device is used as a master terminal device, the master terminal device, the slave terminal device in the vehicle and the domain controller are located in the same local area network, and the master terminal device includes: A monitoring startup module, used to start the UDP monitoring port after the main terminal device is started normally; A communication service startup module, used to start the socket service process of the master terminal device after the master terminal device is started normally; A client identity connection module, used to establish a long connection with the socket service process of the domain controller of the vehicle according to the multicast message monitored by the UDP listening port; the multicast message is sent by the domain controller to the master terminal device in the same local area network, not to the slave terminal device; A rendering module, configured to render on a display interface of the master terminal device according to the vehicle perception and status data received from the socket service process of the domain controller; A server identity connection module, configured to establish a persistent connection with the slave terminal device in response to a request initiated by the slave terminal device in the vehicle to connect to the socket service process of the master terminal device; A data transmission module, for forwarding the vehicle perception and status data to the slave terminal device for independent rendering after successful connection with the slave terminal device; The master terminal device and the slave terminal device are divided into different logical groups, and the vehicle perception and status data are forwarded from the master terminal device to the slave terminal device instead of being distributed to the slave terminal device by the domain controller.

12. A vehicle-mounted terminal device, characterized in that, For performing multi-terminal screen sharing in a vehicle, the vehicle-mounted terminal device is used as a slave terminal device, the slave terminal device is located in the same local area network as the master terminal device and the domain controller in the vehicle, and the slave terminal device includes: A service search module, used to start a network service discovery NSD function to search for a server in the local area network after the slave terminal device is started normally; A network information acquisition module, used for acquiring the IP address of the main terminal device when the socket service process of the main terminal device is turned on and the server end found by the NSD function is the main terminal device; A client identity connection module, used to initiate a request to establish a persistent connection with the main terminal device according to the IP address of the main terminal device; a data receiving module, configured to receive the vehicle perception and status data forwarded from the main terminal device when the socket service connection with the main terminal device is successful; and A rendering module, configured to render the vehicle perception and status data on a display interface of the slave terminal device according to the operating status of the slave terminal device; Among them, the domain controller of the vehicle sends a multicast message to establish a long connection between the socket service process of the domain controller and the master terminal device in the vehicle; the multicast message is sent by the domain controller to the master terminal device in the same local area network, and is not directed to the slave terminal device; the domain controller sends vehicle perception and status data based on the socket service process; the master terminal device and the slave terminal device are divided into different logical groups, and the vehicle perception and status data are forwarded by the master terminal device to the slave terminal device instead of being distributed to the slave terminal device by the domain controller.

13. A system for screen sharing among multiple terminals in a vehicle, characterized in that, include: A domain controller, the domain controller is used to send a UDP multicast message to a target port in the local area network of the vehicle and start a socket service process after normal startup, and is used to establish a long connection with the master terminal device based on the socket to transmit vehicle perception and status data; the multicast message is sent by the domain controller to the master terminal device in the same local area network, not to the slave terminal device; A master terminal device, the master terminal device serves as a client of the domain controller of the vehicle and also as a server of the slave terminal device of the vehicle, and is used to respectively establish a long connection with the domain controller based on a socket and with the slave terminal device based on a socket, and respectively render the vehicle perception and status data received from the domain controller on a display interface of the master terminal device and forward the vehicle perception and status data to the slave terminal device; A slave terminal device, the slave terminal device is used to establish a long connection with the master terminal device based on a socket, and independently render the vehicle perception and status data received from the master terminal device on the display interface of the slave terminal device according to the operating status of the slave terminal device; The master terminal device and the slave terminal device are divided into different logical groups. The vehicle perception and status data are forwarded by the master terminal device to the slave terminal device instead of being distributed to the slave terminal device by the domain controller.

14. The system according to claim 13, characterized in that, The master terminal device and the slave terminal device are display devices built into the vehicle; or, at least one of the master terminal device and the slave terminal device is a display device located in the vehicle and independent of the vehicle.

Images