Method, client, server, device and medium for obtaining multiple data streams

Abstract

The embodiment of the present invention relates to the field of communication technology, and discloses a method for obtaining multiple data streams, a client, a server, a device and a medium. The above-mentioned method for obtaining multiple data streams includes: determining the MIPI interface corresponding to the multiple data streams according to the acquisition request for the multiple data streams; sending a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns a response message carrying the multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction; wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface; receiving the response information returned by the server to obtain the multiple data streams corresponding to the MIPI interface from the response information. The method for obtaining multiple data streams of the embodiment of the present invention can obtain the data streams of multiple cameras through one MIPI interface, so as to better monitor the operation status of the vehicle.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communication technology, and in particular to a method, a client, a server, a device and a medium for acquiring multiple data streams. Background Art

[0002] With the popularization of intelligent automobiles in China, the in-vehicle central control system is becoming more and more important. The current mainstream in-vehicle central control systems are all Android systems, and the camera occupies an important position in the in-vehicle system. By obtaining camera data, the operating status of the vehicle can be monitored.

[0003] The current Android system can only obtain one camera data through one Mobile Industry Processor Interface (MIPI). If you want to obtain multiple camera data, you need to obtain it through multiple MIPI interfaces. However, for some CPUs with fewer MIPI interfaces, it is impossible to obtain more cameras, that is, it is impossible to obtain more camera data and cannot better monitor the vehicle's operating status. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, client, server, device and medium for obtaining multiple data streams, which can obtain data streams of multiple cameras through a MIPI interface, so as to better monitor the operating status of the vehicle.

[0005] To solve the above technical problems, an embodiment of the present invention provides a method for obtaining multiple data streams, which is applied to a client and includes the following steps: determining the MIPI interface corresponding to the multiple data streams based on a request for obtaining the multiple data streams; sending a stream acquisition instruction carrying the MIPI interface to a server, so that the server returns response information carrying the multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction; wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface; receiving the response information returned by the server to obtain the multiple data streams corresponding to the MIPI interface from the response information.

[0006] To solve the above technical problems, an embodiment of the present invention also provides a method for obtaining multiple data streams, which is applied to a server, and includes: receiving a stream acquisition instruction sent by a client; obtaining multiple data streams corresponding to the MIPI interface according to the MIPI interface carried in the stream acquisition instruction, and pre-stored data streams corresponding to each MIPI interface; wherein one MIPI interface corresponds to multiple data streams; and returning response information carrying multiple data streams to the client, so that the client can obtain the multiple data streams from the response information.

[0007] To solve the above technical problems, an embodiment of the present invention also provides a client, including: a determination module, used to determine the MIPI interface corresponding to the multiple data streams according to a request for obtaining the multiple data streams; determine the mobile industry processor interface MIPI corresponding to the multiple data streams; an instruction sending module, used to send a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns response information carrying the multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction; wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface; a processing module, used to receive the response information returned by the server, so as to obtain the multiple data streams corresponding to the MIPI interface from the response information.

[0008] To solve the above technical problems, an embodiment of the present invention also provides a server, including: a receiving module, used to receive a stream acquisition instruction sent by a client; an acquisition module, used to acquire multiple data streams corresponding to the MIPI interface according to the MIPI interface carried in the stream acquisition instruction and the pre-stored data streams corresponding to each MIPI interface; wherein one MIPI interface corresponds to multiple data streams; and a data return module, used to return response information carrying multiple data streams to the client, so that the client can obtain the multiple data streams from the response information.

[0009] To solve the above technical problems, an embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method for acquiring multiple data streams applied to the client, or execute the above method for acquiring multiple data streams applied to the server.

[0010] To solve the above technical problems, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for obtaining multiple data streams applied to a client, or implements the above-mentioned method for obtaining multiple data streams applied to a server.

[0011] Compared with the prior art, the embodiment of the present invention first determines the MIPI interface corresponding to the multiple data streams according to the acquisition request for the multiple data streams, and sends the stream acquisition instruction carrying the MIPI interface to the server, so that the server returns the response information carrying the multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface, and then receives the response information returned by the server to obtain the multiple data streams corresponding to the MIPI interface from the response information. By pre-storing the data streams corresponding to the multiple MIPI interfaces on the server, after determining the MIPI interface corresponding to the requested multiple data streams, the multiple data streams corresponding to the MIPI interface can be obtained from the server, and the multiple data streams are carried in the response information and sent to the client, that is, the data streams of multiple cameras can be obtained through one MIPI interface. Therefore, even on a CPU with fewer MIPI interfaces, the data streams of multiple cameras can be obtained, so as to better monitor the running status of the vehicle.

[0012] In addition, multiple data streams are stored in multiple buffers on the server side respectively; the response information also carries frame information of multiple buffers, and the frame information is used to indicate the camera corresponding to the data stream in the buffer; the multiple data streams corresponding to the MIPI interface are obtained from the response information, including: determining the camera corresponding to each data stream according to the frame information of multiple buffers; based on the camera corresponding to each data stream, obtaining the data stream corresponding to each camera from multiple buffers respectively, so as to determine the camera source of the obtained multiple data streams.

[0013] In addition, before sending a stream acquisition instruction carrying a MIPI interface to the server, so that the server returns a response message carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, it also includes: determining that there are multiple applications on the client that request to obtain multiple data streams at the same time; before receiving the response information returned by the server, so as to obtain the multiple data streams corresponding to the MIPI interface from the response information, it also includes: determining whether any application among the multiple applications terminates the application for multiple data streams; if any application terminates the application for multiple data streams, a stream termination instruction is sent to the server, so that the server terminates the return of the response message carrying multiple data streams to the client based on the stream termination instruction. So that all applications will not receive multiple data streams.

[0014] In addition, after sending a stream acquisition instruction carrying a MIPI interface to the server, so that the server returns a response message carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, the method further includes: if the response message carrying multiple data streams corresponding to the MIPI interface returned by the server is not received within a preset time, then sending a stream termination instruction to the server, so that the server terminates returning the response message carrying multiple data streams corresponding to the MIPI interface to the client based on the stream termination instruction. Timeout detection is performed on the process of the server returning multiple data streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0016] Figure 1 is a framework diagram of a client provided according to an embodiment of the present invention;

[0017] Figure 2 A method for acquiring multiple data streams according to an embodiment of the present invention is provided. Figure 1 ;

[0018] Figure 3 is a state diagram of a MIPI interface provided according to an embodiment of the present invention;

[0019] Figure 4 A method for acquiring multiple data streams according to an embodiment of the present invention is provided. Figure 2 ;

[0020] Figure 5 is a framework diagram of a server provided according to an embodiment of the present invention;

[0021] Figure 6 A method for acquiring multiple data streams according to an embodiment of the present invention is provided. Figure 3 ;

[0022] Figure 7 is an interactive diagram of a method for acquiring multiple data streams provided according to an embodiment of the present invention;

[0023] Figure 8 is a schematic diagram of the structure of a client provided according to an embodiment of the present invention;

[0024] Fig. 9 is a structural diagram of a server provided according to an embodiment of the present invention;

[0025] Fig.10 is a structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present invention, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined and referenced with each other without contradiction.

[0027] One embodiment of the present invention relates to a method for acquiring multiple data streams, including: determining the MIPI interface corresponding to the multiple data streams according to a request for acquiring the multiple data streams; sending a stream acquisition instruction carrying the MIPI interface to a server, so that the server returns response information carrying the multiple data streams corresponding to the MIPI interface to a client according to the stream acquisition instruction; wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface; receiving the response information returned by the server, so as to obtain the multiple data streams corresponding to the MIPI interface from the response information. By pre-storing the data streams corresponding to the multiple MIPI interfaces on the server, after determining the MIPI interface corresponding to the requested multiple data streams, the multiple data streams corresponding to the MIPI interface can be obtained from the server, that is, the data streams of multiple cameras can be obtained through one MIPI interface. Therefore, even on a CPU with fewer MIPI interfaces, the data streams of multiple cameras can be obtained, so as to better monitor the running status of the vehicle.

[0028] The method for acquiring multiple data streams of this embodiment is applied to a client, wherein the client may be a vehicle-mounted device. It is understood that the data streams described in this application refer to camera data streams. In addition, the multiple data streams described in this embodiment are four or fewer data streams.

[0029] The specific process of the method for acquiring multiple data streams in this embodiment is as follows: Figure 1 As shown, including:

[0030] Step 101: Determine MIPI interfaces corresponding to the multiple data streams according to a request for obtaining the multiple data streams.

[0031] Step 102: Send a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns response information carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction.

[0032] Step 103: Receive response information returned by the server to obtain multiple data streams corresponding to the MIPI interface from the response information.

[0033] The implementation details of the method for acquiring multiple data streams in this embodiment are specifically described below. The following content is only provided for easy understanding of the implementation details and is not necessary for implementing this solution.

[0034] In step 101, since the MIPI interface on the CPU in the vehicle system is not unique, each MIPI interface will correspond to the output of multiple data streams, so the client will determine the MIPI interface of the multiple data streams requested for output according to the multiple data streams requested by the user. Among them, the MIPI interface can be a Camera Serial Interface (CSI) interface in MIPI.

[0035] Among them, the user request message carries the identification information of the camera, which is used to indicate the acquisition of data from the camera. The identification information may be the location of the camera. For example, there are four cameras in the vehicle system, namely camera 1, camera 2, camera 3 and camera 4. The user request message carries the identification information of camera 1, camera 2 and camera 3, which means that the user requests to obtain data from camera 1, camera 2 and camera 3. The client queries the corresponding MIPI interface based on the identification information of the three cameras.

[0036] In one example, after the client determines the MIPI interface of the multiple data streams, it first queries the status of the MIPI interface. The status diagram of the MIPI interface is as follows: Figure 2 As shown in the figure, there are four states: Open end, Open running, Close end and Close running. Among them, Open running means that it is in the process of opening, creating resources and opening hardware; Open end means that the open is completed, and it also means that the open is successful; Close running means that it is in the process of closing, closing hardware and releasing resources; Close end means that the close is finished.

[0037] In step 102, if the client obtains that the state of the MIPI interface is the Close end state, it means that the workflow of the previous request for the multiple data streams corresponding to the MIPI interface has been completed, and does not affect the acquisition operation of the current multiple data streams. Then open the MIPI interface, and when the state of the MIPI interface is the Open end state, send a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns a response message carrying the multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface.

[0038] In an example, if the status of the MIPI interface is Open running or Close running, the MIPI interface can be reopened to make the MIPI interface an open end state, so as to send a stream acquisition instruction carrying the MIPI interface to the server, so that the server can return response information carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction.

[0039] In an example, in order to ensure normal communication between the client and the server, it is necessary to obtain the connection status between the client and the server. Therefore, before the client sends a stream acquisition instruction carrying the MIPI interface to the server, it will continuously send heartbeat packets to the server to obtain the connection results fed back by the server. The time interval for sending heartbeat packets is 0.5s.

[0040] If the client fails to send the heartbeat packet, the heartbeat exception callback function is called to feedback to the user layer, and the heartbeat callback pointer is set to null, so that the user performs a close operation. The client calls the close process according to the close operation, that is, it will not send a flow acquisition instruction carrying the MIPI interface to the server.

[0041] In one example, if the client sends a stream fetching instruction carrying a MIPI interface to the server, so that the server returns a response message carrying multiple data streams corresponding to the MIPI interface to the client according to the stream fetching instruction, if the client does not receive the response message returned by the server within the preset time, the timeout callback function is called and fed back to the user layer so that the user performs a close operation, and the client calls the close process according to the close operation, that is, sends an instruction to terminate the fetching of the stream to the server, so that the server terminates the return of the response message to the client based on the instruction to terminate the fetching of the stream. Since a fault may occur during the fetching process, for example, a MIPI interface fault, the client will be provided with a preset time required for fetching the stream, so that the client can perform a timeout detection on the process of returning the response message from the server through the preset time, so that in the case of a timeout in fetching the stream, the instruction to terminate the fetching of the stream is sent to the server in a timely manner.

[0042] In step 103, the client receives the response information returned by the server and carrying the multiple data streams corresponding to the MIPI interface, so as to obtain the requested multiple data streams from the response information.

[0043] Specifically, since multiple data streams are stored in multiple buffers on the server side, and the response information also carries frame information of multiple buffers, where the frame information is used to indicate the camera corresponding to the data stream in the buffer, the client can determine the camera corresponding to each data stream based on the frame information of multiple buffers, and then obtain the data streams corresponding to each camera from multiple buffers based on the camera corresponding to each data stream, so as to determine the camera source of the obtained multiple data streams.

[0044] The frame information may include the ID of the buffer, the ID of each frame image, the corresponding camera channel, the timestamp, etc.

[0045] In one example, the client framework of this embodiment is as follows: Figure 3 As shown, it includes: setting client parameters part, signaling communication part, hardware operation part, heartbeat communication part, and flow control part.

[0046] Among them, the client parameter setting part is used to set various parameters for obtaining multiple data streams when the user's operation is detected, for example, when the user's open operation is detected, that is, when requesting to obtain multiple data streams, specifically including: initialization parameter interface; stream acquisition callback interface; heartbeat abnormality callback interface; stream acquisition callback timeout interface; resolution acquisition interface; resolution setting interface; frame rate interface; detection service MIPI shutdown callback interface.

[0047] The signaling communication part is used for signaling interaction with the server, for example, communicating with the server through the following interfaces, including: querying the MIPI status interface; creating an interface for detecting whether the service MIPI is closed; and releasing the interface for detecting whether the service MIPI is closed.

[0048] The hardware operation part is used to perform hardware operations, including: opening the MIPI interface, closing the MIPI interface, and forcibly closing the MIPI interface.

[0049] The heartbeat communication part is used to perform heartbeat communication with the server, including: creating a heartbeat interface; closing the heartbeat interface.

[0050] The flow control part acquires multiple data streams, including: creating a stream acquisition thread interface; acquiring a buffer and resolution interface; creating a stream acquisition, data callback, and stream acquisition timeout detection interface; closing a stream acquisition thread and releasing an internal resource interface; releasing a buffer interface; and closing all thread interfaces.

[0051] In the specific implementation, the client's stream fetching operation can be found in Figure 3 , there are three specific execution processes:

[0052] (1) If the MIPI status interface is in the Close end state, the specific execution process of the client is: set the initialization parameter interface → set the stream callback interface → set the heartbeat exception callback interface → set the stream callback timeout interface → set the detection service MIPI shutdown callback interface → open the MIPI interface → query the MIPI status interface → create the heartbeat interface → send the stream command to the server → complete.

[0053] (2) If the MIPI status interface is in the Open end state, the specific execution process of the client is: set the initialization parameter interface → set the stream callback interface → set the heartbeat exception callback interface → set the stream callback timeout interface → set the detection service MIPI shutdown callback interface → open the MIPI interface → query the MIPI status interface → create a heartbeat interface → send a stream command to the server → create a stream thread interface → create an interface to detect whether the service MIPI is closed → obtain the buffer and resolution interface → create a stream, data callback and stream timeout detection interface → complete.

[0054] (3) If the MIPI status interface is in Open running or Close running state, the specific execution process of the client is: set initialization parameter interface → set stream acquisition callback interface → set heartbeat abnormality callback interface → set stream acquisition callback timeout interface → set detection service MIPI close callback interface → open MIPI interface → query MIPI status interface → complete.

[0055] Furthermore, after the client sends a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, the user needs to terminate the stream acquisition, or after receiving the multiple data streams returned by the server, the client completes the stream acquisition. At this time, the user will execute a close operation, and the client sends an instruction to terminate the stream acquisition to the server, so that the server can terminate returning the multiple data streams to the client based on the instruction to terminate the stream acquisition.

[0056] The client's abort stream fetching operation can also be seen Figure 3 , there are two specific execution processes:

[0057] (1) Close the stream fetching thread and release the internal resource interface → Close all thread interfaces → Release the buffer interface → Release the interface to detect whether the service MIPI is closed → Close the heartbeat interface → Complete.

[0058] (2) Close the stream fetching thread and release the internal resource interface → Close all thread interfaces → Release the buffer interface → Release the interface for detecting whether the service MIPI is closed → Send a command to terminate the stream fetching to the server → Close the heartbeat interface → Complete.

[0059] Any of the above two execution processes of terminating stream fetching can realize the client's terminating stream fetching operation.

[0060] In this embodiment, the client first determines the MIPI interface corresponding to the multiple data streams according to the acquisition request for the multiple data streams, and sends a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns the response information carrying the multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface, and then receives the response information returned by the server to obtain the multiple data streams corresponding to the MIPI interface from the response information. By pre-storing the data streams corresponding to the multiple MIPI interfaces on the server, after determining the MIPI interface corresponding to the requested multiple data streams, the multiple data streams corresponding to the MIPI interface can be obtained from the server, and the multiple data streams are carried in the response information and sent to the client, that is, the data streams of multiple cameras can be obtained through one MIPI interface. Therefore, even on a CPU with fewer MIPI interfaces, the data streams of multiple cameras can be obtained, so as to better monitor the running status of the vehicle.

[0061] Another embodiment of the present invention relates to a method for obtaining multiple data streams. This embodiment is substantially the same as the first embodiment, except that in this embodiment, there are multiple applications in the client that request to obtain multiple data streams. The specific process of the method for obtaining multiple data streams in this embodiment of the first embodiment is as follows: Figure 4 As shown, including:

[0062] Step 401: Determine MIPI interfaces corresponding to the multiple data streams according to a request for obtaining the multiple data streams.

[0063] Step 401 is substantially the same as step 201 and will not be described in detail herein.

[0064] Step 402: Determine that there are multiple applications on the client that simultaneously request to obtain multiple data streams.

[0065] Step 403: Send a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns response information carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction.

[0066] Specifically, when there are multiple applications on the client that simultaneously request to obtain multiple data streams, that is, there are multiple processes on the client for obtaining multiple data streams, the client can send a stream acquisition instruction carrying the MIPI interface to the server according to the process of any one of the applications, so that the server can return response information carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction.

[0067] Step 404, determine whether any application program among the plurality of application programs has stopped applying for multiple data streams. If any application program has stopped applying for multiple data streams, proceed to step 405; if no application program has stopped applying for multiple data streams, proceed to step 406.

[0068] Step 405: If any application stops applying for multiple data streams, a stop stream acquisition instruction is sent to the server, so that the server stops returning response information carrying multiple data streams to the client based on the stop stream acquisition instruction.

[0069] In an example, if the server stops returning response information carrying multiple data streams to the client based on the instruction to stop fetching the stream, all applications will not receive the response information. It can be seen that this method is the second execution method of the client's operation of stopping fetching the stream in the first embodiment.

[0070] In another example, if any application terminates the application for multiple data streams, the client can only close the process of the application that terminates the application for multiple data streams, so that the application cannot receive the response information returned by the server. Other applications can still receive the response information returned by the server normally. It can be seen that this method is the first execution method of the client's termination of the stream acquisition operation in the first embodiment.

[0071] Step 406: Receive response information returned by the server to obtain multiple data streams corresponding to the MIPI interface from the response information.

[0072] Step 406 is substantially the same as step 203 and will not be described in detail herein.

[0073] This embodiment allows multiple application processes to simultaneously obtain multiple camera data streams, so that different clients can be connected to independently process data, enrich product functions, and thus improve product competitiveness.

[0074] Another embodiment of the present invention relates to a method for obtaining multiple data streams, which is applied to a server. In one example, the framework of the server is as follows: Figure 5As shown, it includes: a signaling module, a heartbeat module, a MIPI resource control module, an openresource module, a close resource module, a frame information sending submodule and a flow acquisition timeout detection module.

[0075] Among them, the signaling module is used for data interaction between the client and the server, such as the client querying the status and the server notifying the client of the status.

[0076] The heartbeat module is used to monitor whether the client is online and receive the heartbeat packet sent by the client. If the client is disconnected, the close resource module is triggered, which stops returning multiple data streams to the client.

[0077] The MIPI resource control module is used to receive a close or open command from the client, that is, a stream terminating command or a stream obtaining command sent by the client.

[0078] The frame information sending submodule is used to return multiple data streams to the client.

[0079] The stream acquisition timeout detection module is used to detect whether the client has timed out from acquiring the stream. If so, the close resource module is triggered to stop returning multiple data streams to the client.

[0080] The specific process of the method for acquiring multiple data streams in this embodiment is as follows: Figure 6 As shown, including:

[0081] Step 601, receiving a stream fetching instruction sent by a client.

[0082] Step 602: According to the MIPI interface carried in the stream acquisition instruction and the pre-stored data streams corresponding to each MIPI interface, multiple data streams corresponding to the MIPI interface are acquired, wherein one MIPI interface corresponds to multiple data streams.

[0083] Step 603: Return response information carrying multiple data streams to the client, so that the client can obtain the multiple data streams from the response information.

[0084] Among them, multiple data streams are respectively stored in multiple buffers of the server, and the response information also carries frame information of multiple buffers, so that the client can determine the camera corresponding to each data stream according to the frame information of multiple buffers, and based on the camera corresponding to each data stream, obtain the data stream corresponding to each camera from multiple buffers respectively; wherein the frame information is used to indicate the camera corresponding to the data stream in the buffer.

[0085] In this embodiment, if the server receives an instruction to stop fetching streams sent by the client, it stops returning response information carrying multiple data streams to the client based on the instruction to stop fetching streams.

[0086] In an example, if the server detects that the client has timed out from fetching the stream, it directly calls the close MIPIresource module to shut down the hardware and release all resources, that is, it stops returning response information to the client; and when it detects that the connection status with the client is abnormal, that is, the heartbeat is abnormal, it calls the close MIPI resource module to shut down the hardware and release all resources, that is, it stops returning response information to the client.

[0087] In this embodiment, after receiving the stream acquisition instruction from the client, the server obtains the multiple data streams corresponding to the MIPI interface through the MIPI interface carried in the stream acquisition instruction and the pre-stored data streams corresponding to each MIPI interface, and returns response information carrying the multiple data streams to the client, so that the client can obtain the multiple data streams from the response information. That is, the acquisition of multiple data streams is achieved through one MIPI interface, which can better monitor the operating status of the vehicle.

[0088] For ease of understanding, we will now Figure 7 The interactive diagram shown illustrates a method for acquiring multiple data streams according to an embodiment of the present invention, which specifically includes:

[0089] S701, the user performs an open operation, that is, the user requests to obtain multiple data streams.

[0090] S702: The client determines the MIPI interface.

[0091] Specifically, the client determines the MIPI interface corresponding to the multiple data streams according to the multiple data streams requested by the user.

[0092] S703: The client sets client parameters.

[0093] The specific configuration is shown in the first embodiment and will not be described in detail here.

[0094] S704, the client queries the status of the MIPI interface.

[0095] S705, the client sends a query instruction to the server.

[0096] Specifically, the client sends a query instruction for querying the status of the MIPI interface to the server, and obtains the status of the MIPI interface for a second time through the server to improve the query accuracy.

[0097] S706, the server queries the status of the MIPI interface and returns the query result to the client.

[0098] S707, the client creates a heartbeat.

[0099] Specifically, the client sends a heartbeat packet to the server every 5 seconds to detect the connection status with the server.

[0100] S708: The server returns the connection result.

[0101] S709, the client opens the MIPI interface and sends an open command to the server.

[0102] When the state of the MIPI interface is the Close end state, the client opens the MIPI interface and sends an open instruction to the server. The open instruction is used to instruct the MIPI resource control module of the server to open the open resource module.

[0103] In one example, if the state of the MIPI interface is Open end, the process proceeds to step S711; if the state of the MIPI interface is Open running or Close running, the user is informed and instructed to reopen or stop opening.

[0104] S710: The server returns a response message carrying an open result.

[0105] S711, the client creates a thread for detecting whether the service MIPI is closed.

[0106] Specifically, the client sends a command to the server to detect whether the service MIPI is closed, so that the server detects whether the service MIPI is closed, that is, detects whether the client needs to stop fetching streams. When only one application on the client requests to obtain multiple data streams, that is, only one fetching thread exists on the client, step S717 is entered.

[0107] S712, the server detects whether the service MIPI needs to be closed and returns the MIPI status.

[0108] S713: The client creates a stream fetching callback thread.

[0109] S714, the server returns a response message carrying multiple data streams corresponding to the MIPI interface.

[0110] S715: The client creates a stream acquisition timeout detection thread.

[0111] S716: The user executes a close operation, that is, the user stops obtaining multiple data streams.

[0112] S717: The client closes the stream fetching thread.

[0113] S718: The client closes the stream fetching timeout detection thread.

[0114] S719, the client releases the detection service MIPI to determine whether to close the thread.

[0115] S720: The client turns off the heartbeat.

[0116] S721, the server turns off the heartbeat.

[0117] S722, the client sends a close command to the server.

[0118] S723, the server closes the hardware resource, i.e., closes resource, and stops returning the response information carrying multiple data streams to the client.

[0119] It should be noted that the above examples in this embodiment are all illustrative for easy understanding and do not constitute a limitation on the technical solution of the present invention.

[0120] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0121] Another embodiment of the present invention relates to a client. The details of the client of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for implementing this embodiment. Figure 8 It is a schematic diagram of the client described in this embodiment, including: a determination module 801, an instruction sending module 802 and a processing module 803.

[0122] Specifically, the determination module 801 is used to determine the MIPI interface corresponding to the multiple data streams according to the acquisition request for the multiple data streams.

[0123] The instruction sending module 802 is used to send a stream acquisition instruction carrying a MIPI interface to the server, so that the server returns a response message carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction. One MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface.

[0124] The processing module 803 is used to receive the response information returned by the server to obtain the multiple data streams corresponding to the MIPI interface from the response information.

[0125] In one example, when multiple data streams are stored in multiple buffers of the server, the processing module 803 is further used to determine the camera corresponding to each data stream according to the frame information of the multiple buffers; based on the camera corresponding to each data stream, the data stream corresponding to each camera is obtained from the multiple buffers. The response information also carries the frame information of the multiple buffers, and the frame information is used to indicate the camera corresponding to the data stream in the buffer.

[0126] It is not difficult to find that this embodiment is an embodiment of the client corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the first embodiment.

[0127] Another embodiment of the present invention relates to a server. The details of the server of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for implementing this example. Fig. 9 It is a schematic diagram of the server described in this embodiment, including: a receiving module 901, an acquisition module 902 and a data return block 903.

[0128] Specifically, the receiving module 901 is used to receive a stream fetching instruction sent by a client.

[0129] The acquisition module 902 is used to obtain multiple data streams corresponding to the MIPI interface according to the MIPI interface carried in the stream acquisition instruction and the pre-stored data streams corresponding to each MIPI interface; wherein one MIPI interface corresponds to multiple data streams; the multiple data streams are respectively stored in multiple buffers on the server side, and the response information also carries frame information of the multiple buffers, so that the client can determine the camera corresponding to each data stream according to the frame information of the multiple buffers, and based on the camera corresponding to each data stream, obtain the data stream corresponding to each camera from the multiple buffers respectively; the frame information is used to indicate the camera corresponding to the data stream in the buffer.

[0130] The data return block 903 is used to return response information carrying multiple data streams to the client, so that the client can obtain the multiple data streams from the response information.

[0131] It is not difficult to find that this embodiment is an embodiment of the server corresponding to the third embodiment, and this embodiment can be implemented in conjunction with the third embodiment. The relevant technical details mentioned in the third embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the third embodiment.

[0132] It is worth mentioning that all modules involved in this embodiment and the fourth embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment and the fourth embodiment do not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment and the fourth embodiment.

[0133] Another embodiment of the present invention relates to an electronic device, such as Fig.10 As shown, it includes: at least one processor 1001; and a memory 1002 that is communicatively connected to the at least one processor 1001; wherein the memory 1002 stores instructions that can be executed by the at least one processor 2001, and the instructions are executed by the at least one processor 1001 so that the at least one processor 1001 can execute the method for obtaining multiple data streams in the above-mentioned embodiments.

[0134] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0135] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0136] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0137] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0138] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A method for acquiring multiple data streams, characterized in that: Applied to the client, including: Determine, according to a request for obtaining multiple data streams, a MIPI interface corresponding to the multiple data streams; Determining that there are multiple applications on the client that simultaneously request to obtain the multiple data streams; Sending a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns a response message carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction; wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface; Determine whether any of the plurality of applications has stopped applying for the multiple data streams; If any of the applications stops applying for the multiple data streams, an instruction to stop obtaining the streams is sent to the server, so that the server stops returning response information carrying the multiple data streams to the client based on the instruction to stop obtaining the streams; If none of the applications terminates the application for the multiple data streams, the response information returned by the server is received through one of the MIPI interfaces to obtain the multiple data streams corresponding to the MIPI interface from the response information.

2. The method for acquiring multiple data streams according to claim 1, characterized in that: The multiple data streams are respectively stored in multiple buffers of the server, and the response information also carries frame information of the multiple buffers, where the frame information is used to indicate the camera corresponding to the data stream in the buffer; The acquiring, from the response information, multiple data streams corresponding to the MIPI interface includes: Determine the camera corresponding to each data stream according to the frame information of the plurality of buffers; Based on the camera corresponding to each data stream, the data stream corresponding to each camera is obtained from the plurality of buffers respectively.

3. The method for acquiring multiple data streams according to claim 1 or 2, characterized in that: After sending a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns response information carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction, the method further includes: If no response information carrying multiple data streams corresponding to the MIPI interface is received from the server within a preset time, an instruction to terminate the stream acquisition is sent to the server, so that the server can stop returning response information carrying the multiple data streams to the client based on the instruction to terminate the stream acquisition.

4. A method for acquiring multiple data streams, characterized in that: Applied to the server, including: Receive streaming instructions sent simultaneously by multiple client applications; According to the MIPI interface carried in the stream acquisition instruction and the pre-stored data streams corresponding to each of the MIPI interfaces, multiple data streams corresponding to the MIPI interfaces are acquired; wherein one MIPI interface corresponds to multiple data streams; If an instruction to stop fetching a stream is received from the client, then based on the instruction to stop fetching a stream, the response information carrying the multiple data streams is stopped from being returned to the client; wherein the instruction to stop fetching a stream is sent by any one of the multiple applications of the client; If the instruction to stop fetching the stream sent by the client is not received, a response message carrying the multiple data streams is returned to the client via one of the MIPI interfaces, so that the client can obtain the multiple data streams from the response message.

5. The method for acquiring multiple data streams according to claim 4, characterized in that: The multiple data streams are respectively stored in multiple buffers of the server, and the response information also carries frame information of the multiple buffers, so that the client can determine the camera corresponding to each data stream according to the frame information of the multiple buffers, and obtain the data stream corresponding to each camera from the multiple buffers based on the camera corresponding to each data stream; The frame information is used to indicate the camera corresponding to the data stream in the buffer.

6. A client, characterized in that: include: A determination module, configured to determine a MIPI interface corresponding to the multiple data streams according to a request for obtaining the multiple data streams; It is also used to determine that there are multiple applications on the client that simultaneously request to obtain the multiple data streams; An instruction sending module, used for sending a stream acquisition instruction carrying the MIPI interface to the server, so that the server returns a response message carrying multiple data streams corresponding to the MIPI interface to the client according to the stream acquisition instruction; wherein one MIPI interface corresponds to multiple data streams, and the server pre-stores the data streams corresponding to each MIPI interface; A processing module is used to determine whether any of the multiple applications terminates the application for the multiple data streams; if any of the applications terminates the application for the multiple data streams, a command to terminate the acquisition of the stream is sent to the server, so that the server terminates returning the response information carrying the multiple data streams to the client based on the instruction to terminate the acquisition of the stream; if none of the applications terminates the application for the multiple data streams, the response information returned by the server is received through one of the MIPI interfaces, so as to obtain the multiple data streams corresponding to the MIPI interface from the response information.

7. A server, characterized in that: include: A receiving module, used for receiving stream fetching instructions sent simultaneously by multiple applications of the client; An acquisition module, configured to acquire multiple data streams corresponding to the MIPI interface according to the MIPI interface carried in the stream acquisition instruction and the pre-stored data streams corresponding to each of the MIPI interfaces; wherein one MIPI interface corresponds to multiple data streams; A data return module, which is used to, if a command to terminate stream acquisition is received from the client, then based on the command to terminate stream acquisition, stop returning response information carrying the multiple data streams to the client; wherein the command to terminate stream acquisition is sent through any one of the multiple applications of the client; if no command to terminate stream acquisition is received from the client, then return response information carrying the multiple data streams to the client through one of the MIPI interfaces, so that the client can obtain the multiple data streams from the response information.

8. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for acquiring multiple data streams as described in any one of claims 1 to 3, or execute the method for acquiring multiple data streams as described in claim 4 or 5.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for acquiring multiple data streams according to any one of claims 1 to 3 is implemented, or the method for acquiring multiple data streams according to claim 4 or 5 is implemented.

Images