Screen sharing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a screen sharing method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of cloud technologies, cloud conferences, intelligent transportation and the like. When the sharing instruction is received, the sub-processes can be started through the main process, and the communication connection between the processes is established; performing multimedia data acquisition through the sub-process screen display page to obtain screen multimedia data; the screen multimedia data comprises screen picture data and screen audio data; therefore, recording of pictures and audios of the screen is realized. The screen multimedia data is sent to the host process based on communication between the processes, and the host process transmits the screen multimedia data to the server, so that the screen display page is shared to each session object; as the online video session is realized by the host process, the services of the to-be-shared screen and the host process can be transmitted based on one session instance; the to-be-shared screen does not need to be screenshot frame by frame, the performance consumption is reduced, and the screen sharing practicability is effectively improved.

Description

Technical Field

[0001] The present application relates to technical fields such as cloud technology, cloud conferencing, and smart transportation. The present application relates to a screen sharing method, device, electronic device, storage medium, and program product. Background Art

[0002] With the development of Internet technology, screen sharing functions are available in some online scenarios. For example, in a multi-person online conference scenario, conference participants can share the current screen of their devices with others in the conference.

[0003] In the related art, a real-time screenshot of the current screen of a device can be taken within the process of an application program; and then the real-time screenshot of the current screen is sent to conference participants in an online conference, thereby completing screen sharing.

[0004] However, the above method is inefficient and occupies a high CPU, which easily affects device performance, so the practicality of the above screen sharing is poor. Summary of the invention

[0005] The present application provides a screen sharing method, device, electronic device, storage medium and program product, which can solve the problem of poor practicality of screen sharing in related technologies. The technical solution is as follows:

[0006] In one aspect, a screen sharing method is provided, the method comprising:

[0007] In response to a sharing instruction triggered by a screen sharing control in a target session page, a subprocess corresponding to the sharing instruction is started through a main process corresponding to the online video session, and a communication connection is established between the main process and the subprocess; the sharing instruction instructs sharing the screen display page of the target session object to each session object in the online video session;

[0008] Collecting multimedia data of the screen display page through the subprocess to obtain screen multimedia data, wherein the screen multimedia data includes screen picture data and screen audio data corresponding to the screen display page;

[0009] Based on the communication connection between the main process and the sub-process, sending the screen multimedia data to the main process;

[0010] The screen multimedia data is transmitted to the server through the main process, so that the screen display page is shared to each session object in the online video session through the server.

[0011] In another aspect, a screen sharing device is provided, the device comprising:

[0012] A sub-process starting module, configured to respond to a sharing instruction triggered by a screen sharing control in a target session page, start a sub-process corresponding to the sharing instruction through a main process corresponding to the online video session, and establish a communication connection between the main process and the sub-process; the sharing instruction instructs sharing the screen display page of the target session object to each session object in the online video session;

[0013] A sub-process data collection module, used for collecting multimedia data of the screen display page through the sub-process to obtain screen multimedia data, wherein the screen multimedia data includes screen picture data and screen audio data corresponding to the screen display page;

[0014] An inter-process data sending module, used for sending the screen multimedia data to the main process based on the communication connection between the main process and the sub-process;

[0015] The transmission module is used to transmit the screen multimedia data to the server through the main process, so as to share the screen display page to each session object in the online video session through the server.

[0016] In one possible implementation, the device further includes:

[0017] A startup module, used for displaying a screen sharing startup page through the main process, wherein the screen sharing startup page displays a startup control for triggering the startup of screen sharing;

[0018] The sub-process starting module is used to start the sub-process corresponding to the sharing instruction through the main process in response to the triggering operation of the start control in the screen sharing start page.

[0019] In a possible implementation, the screen sharing startup page further includes a selection control, and the selection control is used to trigger a selection operation for at least one candidate page on the screen;

[0020] The sub-process startup module is used to respond to a selection operation triggered by a selection control and a trigger operation on a startup control, use the page selected based on the selection operation as the screen display page, and start a sub-process for collecting multimedia data for the screen display page through the main process.

[0021] In a possible implementation, an object picture corresponding to the target conversation object is displayed in the target conversation page;

[0022] The device also includes:

[0023] A main process recording module, used for recording an object screen corresponding to a target conversation object through the main process to obtain conversation object data corresponding to the target conversation object;

[0024] The transmission module is used to transmit the screen multimedia data and the session object data to the server through the main process.

[0025] In a possible implementation, the sub-process data collection module is used to:

[0026] Collecting multimedia data from the screen display page through the subprocess;

[0027] For the collected original picture data of the video type, compressing the original picture data to obtain the screen picture data of the screen display page;

[0028] For the collected original audio data of the audio type, the original audio data is used as the screen audio data corresponding to the screen display page.

[0029] In a possible implementation, the conversation object data includes object picture data and object audio data corresponding to the target conversation object;

[0030] The transmission module, when transmitting the screen multimedia data and the session object data to the server through the main process, is specifically used to:

[0031] By means of the main process, the object audio data and the screen audio data are aligned based on the timestamps of the object audio data and the screen audio data, and the aligned object audio data and the screen audio data are compressed, and the compressed object audio data and the screen audio data are sent to the server;

[0032] Through the main process, the object picture data and the screen picture data are aligned based on the timestamps of the object picture data and the screen picture data, and the aligned object picture data and the screen picture data are sent to the server.

[0033] In one possible implementation, the device further includes:

[0034] A business data acquisition module, used to acquire business log data corresponding to the multimedia data collection process of the screen display page through the sub-process, and send the business log data to the main process;

[0035] The service detection module is used to respond to the service detection instruction for the online video session and perform service detection on the multimedia data collection process of the sub-process based on the service log data through the main process.

[0036] In one possible implementation, the device further includes:

[0037] A creation module, used to create a socket communication end of the main process based on the communication address and port of the main process through the main process;

[0038] The sub-process startup module is specifically used to establish a communication connection between the main process and the sub-process:

[0039] A socket communication terminal of the sub-process is created through the sub-process, and a communication connection is established between the socket communication terminal of the main process and the socket communication terminal of the sub-process based on the communication address and port of the main process.

[0040] In one possible implementation, the device further includes:

[0041] A display module, configured to display the screen display page and an end control in the target session page;

[0042] The end module is used to respond to the triggering operation of the end control and send an end notification message to the sub-process through the main process to make the sub-process terminate the multimedia data collection of the screen display page.

[0043] On the other hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-mentioned screen sharing method.

[0044] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned screen sharing method is implemented.

[0045] On the other hand, a computer program product is provided, including a computer program, wherein the computer program implements the above-mentioned screen sharing method when executed by a processor.

[0046] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0047] The screen sharing method provided by the present application, when receiving a sharing instruction triggered by a screen sharing control in a target session page, can start a subprocess corresponding to the sharing instruction through a main process, and establish a communication connection between the main process and the subprocess; the subprocess can collect multimedia data for the screen display page to be shared, and obtain screen multimedia data, which includes screen picture data and screen audio data of the screen display page; thereby, the screen and audio of the screen to be shared can be recorded through the subprocess. Moreover, the screen multimedia data is sent to the main process based on the communication connection between processes, and the main process transmits the screen multimedia data to the server, so that the server shares the screen display page to each session object; and since the online video session is implemented by the main process, the screen to be shared and the business of the main process are transmitted based on a session instance; there is no need to take screenshots of the shared screen frame by frame, which reduces performance consumption and CPU occupancy, does not affect the performance of the device, and can simultaneously share the audio and video data of the screen, thereby effectively improving the practicality of screen sharing. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in describing the embodiments of the present application.

[0049] Figure 1 A schematic diagram of an implementation environment of a screen sharing method provided in an embodiment of the present application;

[0050] Figure 2 A flowchart of a screen sharing method provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of a session page for an online conference scenario provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of a session page for an online conference scenario provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of a session page of an online conference scenario provided by an initiator in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of a session page for an online conference scenario of a viewer provided in an embodiment of the present application;

[0055] Figure 7 A screen sharing process diagram provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the structure of a screen sharing device provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0059] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0060] It is understandable that in the specific implementation of the present application, when the above embodiments of the present application are applied to specific products or technologies, any object-related data, such as screen image data, screen audio data, object image data of the target session object, object audio data, the address and port of the main process, etc., need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if any object-related data is involved in the above techniques in the embodiments of the present application, these data need to be obtained with the authorization and consent of the object and in compliance with the relevant laws, regulations and standards of the country and region.

[0061] Figure 1 A schematic diagram of an implementation environment of a screen sharing method provided in this application. Figure 1 As shown, the implementation environment includes: a server 101 and a terminal 102. The server 101 may be a background server of an application. The terminal 102 is installed with an application, and the terminal 102 and the server 101 may perform data interaction based on the application.

[0062] In this application, the application has a screen sharing function. The screen sharing function is a function of sharing a screen display page with a conversation partner in the online video conversation when the application is conducting an online video conversation.

[0063] In one possible scenario, the online video session may be a live video broadcast. In the live broadcast room, the anchor can share the screen display page of the anchor's device with each audience in the live broadcast room. For example, the game screen of the game being played on the screen of the anchor's device, the product details page of the product being sold, etc. Correspondingly, each terminal 102 may be installed with a live broadcast application, and the server 101 is the background server of the live broadcast application. The anchor's terminal 102 can synchronize the live broadcast room screen to each audience in the live broadcast room, as well as the game screen shared by the anchor on the screen.

[0064] In another possible scenario, the online video session may be an online conference. For example, in a multi-person online conference, a conference participant may share the screen display page of the device where the conference participant is located with other participants in the conference. For example, a page of a PPT or an office document page played on the screen of the device where the conference participant is located. Accordingly, each terminal 102 may be installed with an office application supporting online conferences, a cloud conference application, etc. While the conference participant is having a cloud conference with other participants in the online conference, he or she may also share the page of the PPT displayed on his or her terminal 102 with other participants.

[0065] The server 101 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The above-mentioned network may include, but is not limited to: wired network, wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, Wi-Fi and other networks that realize wireless communication. The terminal 102 may be a smart phone (such as Android mobile phone, iOS mobile phone, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Devices), a PDA (personal digital assistant), a desktop computer, a vehicle terminal (such as a vehicle navigation terminal, a vehicle computer, etc.), a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication, but are not limited thereto. The specific may also be determined based on the actual application scenario requirements, and is not limited here.

[0066] The screen sharing method provided in this application involves technical fields such as cloud technology, cloud conferencing, cloud education, and smart transportation. For example, the above-mentioned screen sharing method can be applied to cloud conferencing technology. For example, during a multi-person online conference, the PPT page displayed on the screen of the device where the conference participant is located is shared with other conference participants in the conference. For another example, the above-mentioned screen sharing method can also be applied to cloud education scenarios, for example, the courseware displayed on the teacher's terminal can be shared with each student participating in the cloud education course.

[0067] It is understandable that cloud conferencing is an efficient, convenient and low-cost form of conferencing based on cloud computing technology. Users only need to use the Internet interface and perform simple and easy-to-use operations to quickly and efficiently share voice, data files and videos with teams and customers around the world, while the cloud conferencing service provider helps users operate complex technologies such as data transmission and processing during the meeting.

[0068] At present, cloud conferencing in my country mainly focuses on service content based on the SaaS (Software as a Service) model, including telephone, network, video and other service forms. Video conferencing based on cloud computing is called cloud conferencing.

[0069] In the era of cloud conferencing, data transmission, processing, and storage are all handled by the computer resources of video conferencing manufacturers. Users no longer need to purchase expensive hardware or install cumbersome software. They only need to open a browser and log in to the corresponding interface to conduct efficient remote meetings.

[0070] The cloud conference system supports dynamic cluster deployment of multiple servers and provides multiple high-performance servers, which greatly improves the stability, security and availability of conferences. In recent years, video conferencing has been welcomed by many users because it can greatly improve communication efficiency, continuously reduce communication costs, and bring about an upgrade in internal management level. It has been widely used in various fields such as transportation, finance, operators, education, enterprises, and Internet of Vehicles. There is no doubt that after the use of cloud computing, video conferencing has a stronger appeal in terms of convenience, speed, and ease of use, which will surely stimulate the arrival of a new wave of video conferencing applications.

[0071] It is understandable that Cloud Computing Education (CCEDU) refers to an education platform service based on the cloud computing business model. On the cloud platform, all educational institutions, training institutions, enrollment service institutions, publicity institutions, industry associations, management institutions, industry media, legal structures, etc. are integrated into a resource pool. Each resource is displayed and interacted with each other, and communication is carried out on demand to reach an agreement, thereby reducing education costs and improving efficiency.

[0072] It is understandable that Cloud Social is a virtual social application model of interactive applications of the Internet of Things, cloud computing and mobile Internet. It aims to establish the famous "resource sharing relationship map" and then carry out online social networking. The main feature of Cloud Social is to integrate and evaluate a large number of social resources to form an effective resource pool to provide services to users on demand. The more users who participate in sharing, the greater the utilization value that can be created.

[0073] It is understandable that cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0074] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0075] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0076] Figure 2 The following is a flow chart of a screen sharing method provided in an embodiment of the present application. The execution subject of the method may be an electronic device. Figure 2 As shown, the method includes the following steps.

[0077] Step 201: In response to a sharing instruction triggered by a screen sharing control in a target session page, the electronic device starts a sub-process corresponding to the sharing instruction through a main process corresponding to the online video session, and establishes a communication connection between the main process and the sub-process.

[0078] The sharing instruction indicates that the screen display page in the current device is shared to each session object in the online video session. The target session page is a page displayed in the online video session on the side of the target session object. The main process is a process for implementing the online video session, for example, the main process can be used to execute the display step of the target session page.

[0079] Exemplarily, the target conversation page may be a page of an online video conversation being conducted by the target conversation object. The online video conversation may be any form of conversation that supports a single person or multiple people to conduct an online conversation in the form of video. For example, the online video conversation may include, but is not limited to: a live video initiated on a live broadcast application, an online conference conducted on an office application, an instant video chat in a social application, or an online education video on a cloud education platform. This application does not limit the form of the online video conversation, the number of participants in the conversation, etc.

[0080] In one possible mode, the target conversation page displays an object picture corresponding to the target conversation object. The object picture is obtained by the main process collecting multimedia data, and the main process can also transmit the collected object picture to the server so that each conversation object in the online video conversation can see the object picture corresponding to the target conversation object.

[0081] The object screen corresponding to the target conversation object may be a screen obtained by directly collecting multimedia data from the target conversation object; or, it may be a screen obtained by collecting multimedia data from the collected object specified by the target conversation object. For example, a screen for collecting data from the surrounding environment where the target conversation object is located, or from the slides being presented. The target conversation object may collect data from the required collection object by controlling the camera of the electronic device, and synchronize the data to each conversation object. The present application does not limit the specific content of the object screen.

[0082] In this step, a screen sharing control may be displayed in the target session page. The target session object may trigger the screen sharing control to trigger the sharing instruction. When receiving a sharing instruction triggered by the screen sharing control, the electronic device may start a sub-process through the main process corresponding to the online video session, and the sub-process may be used to execute the screen capture process corresponding to the sharing instruction. Among them, the main process may be the parent process of the sub-process. In addition, the electronic device may also establish a communication connection between the sub-process and the main process, so that the screen capture data of the sub-process can be sent to the main process for processing and transmission.

[0083] In a possible implementation, when the sharing instruction is detected, the electronic device may first invoke a screen sharing startup page for starting the screen sharing process through the main process; and may start the sub-process through the screen sharing startup page. Exemplarily, before starting the sub-process corresponding to the sharing instruction through the main process corresponding to the online video session, the following steps are also included:

[0084] The electronic device displays a screen sharing startup page through the main process, wherein a startup control for triggering the startup of screen sharing is displayed on the screen sharing startup page;

[0085] Accordingly, the main process corresponding to the online video session starts the sub-process corresponding to the sharing instruction, including the following steps:

[0086] In response to a triggering operation on a start control in the screen sharing start page, the electronic device starts a sub-process corresponding to the sharing instruction through a main process.

[0087] like Figure 3 As shown, an online video session is used as an example for online conference. Figure 3 In the online conference page shown in FIG. 1 , the screen sharing control may be a presentation screen button. When the presentation screen button is triggered, the user may enter the following Figure 4 The screen sharing startup page shown. Of course, the online video session page may also include but is not limited to: a camera button for turning on the camera, a flip camera button for flipping the camera, a beauty button, a hang-up button for hanging up the current video session, a microphone button for turning the microphone on or off, a button for turning the speaker on or off, etc. Among them, the online video session page also includes the video images of each participant in the online video conference, which can be the video images collected by the devices of each participant.

[0088] like Figure 4 As shown, the start control may be a start live button in the screen sharing start page. In addition, the screen sharing start page may also include a screen display page to be screen shared. For example, the screen display page may be a certain APP page running in the electronic device. When the user triggers the start live button, the sub-process may be started so as to use the sub-process to perform the data collection process for the screen display page.

[0089] In a possible implementation, the screen sharing startup page also includes a selection control, through which a desired page can be selected from multiple candidate pages for sharing. Exemplarily, the screen sharing startup page also includes a selection control, which is used to trigger a selection operation for at least one candidate page on the screen. Accordingly, the electronic device starts a subprocess corresponding to the sharing instruction through a main process corresponding to the online video session, which may include the following steps:

[0090] In response to a selection operation triggered by a selection control and a trigger operation on a start control, the electronic device uses the page selected based on the selection operation as the screen display page, and starts a subprocess for collecting multimedia data for the screen display page through the main process.

[0091] Among them, when the electronic device detects a trigger operation on the screen sharing control, it can display the screen sharing startup page and display multiple candidate pages in the screen sharing startup page; when a selection operation triggered among the multiple candidate pages is detected, it can be used as a page to be screen shared, and a sub-process for the screen display page can be started to start multimedia data collection for the screen display page in the sub-process.

[0092] In a possible implementation, the electronic device can realize the communication connection between processes through the socket technology. Exemplarily, the communication connection establishment process includes the following steps:

[0093] Before starting the sub-process corresponding to the sharing instruction through the main process corresponding to the online video session, the electronic device creates a socket communication end of the main process based on the communication address and port of the main process through the main process;

[0094] Accordingly, establishing a communication connection between the main process and the sub-process includes the following steps:

[0095] The electronic device creates a socket communication end of the sub-process through the sub-process, and establishes a communication connection between the socket communication end of the main process and the socket communication end of the sub-process based on the communication address and port of the main process.

[0096] Exemplarily, when the electronic device detects a sharing instruction, it can first start the port detection of the local socket communication terminal in the main process. Among them, the electronic device can initiate a socket communication terminal whose address and port are the local address and the preconfigured designated port respectively through the GCDAsyncSocket network encapsulation library. The local address can be the local IP address of the electronic device, such as 127.0.0.1; thereby starting the detection status of the communication terminal of the main process. After the child process is started, a socket communication terminal can be created in the child process, and based on the communication address and port of the main process, an attempt is made to communicate with the main process. The main process can detect the connection status of the socket communication terminal in real time; when a connection request of the socket communication terminal of the child process is detected, a socket-based communication connection between the main process and the child process can be established after verification.

[0097] In one possible approach, the screen sharing function in the online video session can be implemented based on the ReplayKit technology of the iOS system. The child process can be a ReplayKit child process extended in the main process using the ReplayKit technology.

[0098] For example, when the detection state of the communication end of the main process is started, the electronic device can use the system class RPSystemBroadcastPickerView to realize the invocation and display of the Replaykit live screen interface, that is, to invoke and display the screen sharing startup page. The sub-process can be a sub-process extended by Replaykit technology for realizing screen capture of the Replaykit live screen interface. When the user clicks the start live button on the Replaykit live screen interface, the electronic device can start a Replaykit extended sub-process using Replaykit technology. In order to execute the multimedia data acquisition process of the screen display page through the sub-process, that is, to execute the following step 202.

[0099] Step 202: The electronic device collects multimedia data from the screen display page through the sub-process to obtain screen multimedia data.

[0100] The screen multimedia data includes screen image data and screen audio data of the screen display page. The screen audio data may be the corresponding audio during the display process of the screen display page; for example, the sound effect when a button is triggered in the screen display page, the sound in the video played in the screen display page, and other audio of any form of content associated with the screen display page.

[0101] Exemplarily, the system audio corresponding to the screen display page during the display process can be used as the screen audio data. In this step, the electronic device can collect the screen data of the screen display page through the subprocess to obtain the screen picture data; and collect the system audio of the screen display page to obtain the screen audio data.

[0102] In a possible implementation, the electronic device may selectively encode and compress different types of multimedia data using the types of the collected multimedia data to obtain screen multimedia data. Accordingly, the implementation of step 202 may include:

[0103] The electronic device collects multimedia data from the screen display page through the subprocess;

[0104] For the collected original picture data of the video type, the electronic device compresses the original picture data to obtain the screen picture data of the screen display page;

[0105] For the collected original audio data of the audio type, the electronic device uses the original audio data as screen audio data corresponding to the screen display page.

[0106] Exemplarily, the electronic device can use the method of the system RPBroadcastHandler class to realize the multimedia data collection of the screen display page. For example, the electronic device can obtain the screen multimedia data based on the returned data by calling the following function:

[0107] -(void)processSampleBuffer:(CMSampleBufferRef)sampleBufferwithType:(RPSampleBufferType)sampleBufferType;

[0108] If the returned type is RPSampleBufferTypeAudioApp, it is audio data, and the electronic device can directly use the returned PCM raw data as screen audio data;

[0109] If the returned type is RPSampleBufferTypeVideo, it is video data. The electronic device can first encode and compress the returned original picture data through the H265 / H264 hardware encoder provided by the system VideoToolBox, and use the compressed data as the screen picture data of the screen display page.

[0110] It should be noted that for audio data, since the data volume is small, the directly collected original audio data can be sent to the main process through the connected socket connection. For screen image data, since the data volume of the original screen is large, the efficiency of sending it directly to the main process is low. The H265 / H264 hardware encoder provided by the system VideoToolBox is first encoded and compressed, and the compressed data volume is greatly reduced. After compression, it is sent to the main process through the socket connection, which can effectively improve the data processing efficiency. In this way, different processing methods for data types are realized, and the actual processing efficiency is improved as much as possible while ensuring stability and reliability, which improves the fluency of the subsequent screen sharing process.

[0111] Step 203: The electronic device sends the screen multimedia data to the main process based on the communication connection between the main process and the sub-process.

[0112] In this step, the child process can send the screen multimedia data to the main process through the socket communication connection in the connected state.

[0113] In a possible implementation, the electronic device may also send the service log of the sub-process to the main process, and the main process performs subsequent processing such as unified printing or service detection. Accordingly, the process may include the following steps:

[0114] The electronic device obtains the business log data corresponding to the multimedia data collection process of the screen display page through the sub-process, and sends the business log data to the main process;

[0115] The electronic device responds to the service detection instruction for the online video session by performing service detection on the multimedia data collection process of the sub-process based on the service log data through the main process.

[0116] For example, the business log data may include, but is not limited to: the frame rate and bit rate used by the sub-process for multimedia data collection, and the QoS (Quality of Service) corresponding to the collection process, etc. The business detection may be used to troubleshoot business problems for the multi-media data collection process. For example, the frame rate and bit rate corresponding to the sub-process data collection process may be periodically detected, and the frame rate or bit rate may be adjusted in real time based on the transmission quality between the sub-process and the main process. For example, when the system performance is low or the transmission rate is slow, or even the transmission is interrupted, the frame rate and bit rate may be reduced; conversely, the frame rate and bit rate may be increased.

[0117] In addition, a comprehensive judgment can be made in combination with the business log data. For example, the service quality corresponding to the online video session and whether congestion problems occur can be judged by combining the business log data of the main process and the business log data sent by the child process.

[0118] It should be noted that by sending the business log data of the sub-process to the main process for unified printing or business troubleshooting, the business pressure of the sub-process can be reduced, and the management and control of the main process over the sub-process can be improved, thereby ensuring the reliability and stability of screen sharing.

[0119] Step 204: The electronic device transmits the screen multimedia data to the server through the main process, so as to share the screen display page with each session object in the online video session through the server.

[0120] In this step, the electronic device transmits the screen multimedia data and the service data of the main process corresponding to the online video session to the server through the main process. The server transmits the screen multimedia data to each session object so that the device where each session object is located can display the screen display page in the online video session.

[0121] In a possible implementation, the target conversation page displays an object picture corresponding to the target conversation object; the electronic device can also acquire the object picture through the main process. The process may include the following steps:

[0122] The electronic device records the object screen corresponding to the target conversation object through the main process to obtain conversation object data corresponding to the target conversation object;

[0123] Accordingly, in step 204, transmitting the screen multimedia data to the server through the main process may include the following steps:

[0124] The electronic device transmits the screen multimedia data and the session object data to the server through the main process.

[0125] The object picture may be a picture collected from the target conversation object, or may be a picture collected from a designated object to be collected, for example, a picture of the surrounding environment of the target conversation object, a picture of a demonstrated slide, etc.

[0126] In this step, the electronic device transmits the screen multimedia data and the session object data to the server as the data to be transmitted for the current online video session through the main process. The server transmits the screen multimedia data and the session object data to each session object so that the device where each session object is located can display the object screen and the screen display page.

[0127] In a possible implementation, the audio data collected by the sub-process and the audio data collected by the main process can be aligned through the main process to ensure the accuracy of screen sharing in the online video session. Accordingly, the session object data includes the object screen data and object audio data corresponding to the target session object; in step 204, transmitting the screen multimedia data and the session object data to the server through the main process may include the following steps:

[0128] Through the main process, the object audio data and the screen audio data are aligned based on the timestamps of the object audio data and the screen audio data, and the aligned object audio data and the screen audio data are compressed, and the compressed object audio data and the screen audio data are sent to the server;

[0129] Through the main process, the object picture data and the screen picture data are aligned based on the timestamps of the object picture data and the screen picture data, and the aligned object picture data and the screen picture data are sent to the server.

[0130] Among them, the screen audio data can be the original data that has not been encoded and compressed by the sub-process; through the main process, the aligned screen audio data and the object audio data collected by the main process can be uniformly encoded and compressed, and then the two channels of audio data after the encoding and compression processing can be uniformly sent to the server.

[0131] For the screen image data, if the screen image data is obtained by encoding and compressing the collected original image data through a sub-process, the main process can directly align the screen image data with the object image data and send them uniformly.

[0132] It should be noted that the audio from the screen is aligned with the audio from the object through the main process, and then encoded and compressed for unified transmission to the server, thereby effectively ensuring the accuracy of the data and the accuracy of the audio during subsequent playback, thereby improving the actual processing efficiency of screen sharing.

[0133] In a possible implementation, the electronic device may display a screen display page being shared; wherein, during the screen sharing process, the user may also trigger the end of the sharing process as needed. The process may include the following steps:

[0134] The electronic device displays the screen display page and the end control in the target session page;

[0135] In response to the triggering operation of the end control, the electronic device sends an end notification message to the sub-process through the main process, so that the sub-process terminates the multimedia data collection of the screen display page.

[0136] The end notification message is used to indicate the termination of the multimedia data collection process for the screen display page. Since an end control for triggering the end of sharing is displayed in the target session page, the user can end the screen sharing by triggering the end control. For example, if it is detected that the end control is triggered, the main process notifies the Replaykit extension child process through CFNotificationCenterGetDarwinNotifyCenter, so that the child process uses finishBroadcastWithError to terminate the multimedia data collection process for the screen display page.

[0137] like Figure 5 As shown, Figure 5 , a schematic diagram of a screen display page of screen sharing in progress in the initiator's device is shown. For the initiator of screen sharing, the screen display page of screen sharing in progress can be displayed. Among them, the end control can be an end presentation button in the session page.

[0138] like Figure 6 As shown, Figure 6 FIG. 1 shows a schematic diagram of a screen display page shared by the initiator in the device of the viewer. Figure 6 As shown, the viewer can now see the screen interface shared by the initiator and the original camera image at the same time, that is, the image captured by the main process in the initiator's device; if the initiator shares the system audio and turns on the microphone, the viewer can hear the initiator's system audio and microphone audio at the same time, for a total of two channels of sound.

[0139] It should be noted that by uniformly transmitting screen multimedia data and session object data as the data to be transmitted in the current session, the screen display page and the target session page are unified into one session for processing, which can reduce the consumption of device performance, has high stability, and is not easy to trigger the system's memory limit on child processes.

[0140] Figure 7 This is a flow chart of a process of initiating screen sharing in an online video session of a client provided by an embodiment of the present application. Figure 7 The process shown further introduces the screen sharing method of this application.

[0141] like Figure 7 As shown, the screen sharing method of the present application may include the following process:

[0142] 1. The target conversation object acts as the initiator and can initiate screen sharing;

[0143] 2. The main process starts the local socket port to detect the connection status; for example, the GCDAsyncSocket network encapsulation library can be used to initiate a socket end with the local address and the specified port number;

[0144] 3. Call up the Replaykit live screen interface and display it;

[0145] 4. The user clicks the Start Live button on the Replaykit live screen interface to start the collection sub-process based on the Replaykit technology extension.

[0146] 5. Start a socket in the Replaykit child process to connect to the port of the main process to establish a communication connection between the two processes, so as to realize the network data sending and receiving between the two processes based on the communication connection;

[0147] 6. Screen audio and video data collection through Replaykit extended child process;

[0148] 7. For audio data, the sub-process can directly send it to the main process through the socket connection; for the original picture data of the video type, the sub-process encodes and compresses the original picture data through the hard encoder to obtain the screen picture data, and sends the screen picture data to the main process through the socket connection;

[0149] In addition, the sub-process can also send the business logs of the sub-process during the data collection process, such as frame rate, bit rate and other log data, to the main process; the main process combines its own business log data with the business log data of the sub-process to uniformly perform subsequent printing, business detection and other processing processes.

[0150] 8. The main process sends the screen multimedia data and session multimedia data to the server based on one session;

[0151] The main process may also align the screen audio data and the object audio data, encode and compress them in sequence, and transmit the processed screen audio data and the object audio data to the server. In addition, the screen picture data and the object picture data may be aligned, and the processed screen picture data and the object picture data may be transmitted to the server.

[0152] 9. The shared screen display page can be synchronously displayed in the electronic device of the target conversation object. For example, the screen display page is displayed in the target conversation page; of course, the target conversation page can also provide an end control, such as an end screen presentation button, an end sharing button, etc.

[0153] If it is detected that the screen sharing is terminated by triggering the end control, an end notification message can be sent to the child process through the main process, so that the child process terminates the live broadcast extension of the Replaykit interface.

[0154] It should be noted that in this application, a simple business model can be used to effectively isolate the impact of screen sharing data flow on the performance of the application where the online video session is located; at the same time, the picture content and picture sound of the screen to be shared can be collected at the same time, realizing the sharing of screen content outside the application interface, and efficiently meeting the needs of stable screen audio and video sharing in scenarios such as live broadcasts or meetings in the iOS system; and, the Replaykit extension sub-process has no business logic coupling, and only needs to collect and process the data of the shared screen, which is uniformly processed in a role session of the main process, with low and stable consumption of device performance, and is not easy to trigger the system's memory limit on the sub-process, thereby effectively improving the practicality of screen sharing.

[0155] The screen sharing method provided by the present application, when receiving a sharing instruction triggered by a screen sharing control in a target session page, can start a subprocess corresponding to the sharing instruction through a main process, and establish a communication connection between the main process and the subprocess; the subprocess can collect multimedia data for the screen display page to be shared, and obtain screen multimedia data, which includes screen picture data and screen audio data of the screen display page; thereby, the screen and audio of the screen to be shared can be recorded through the subprocess. Moreover, the screen multimedia data is sent to the main process based on the communication connection between processes, and the main process transmits the screen multimedia data to the server, so that the server shares the screen display page to each session object; and since the online video session is implemented by the main process, the screen to be shared and the business of the main process are transmitted based on a session instance; there is no need to take screenshots of the shared screen frame by frame, which reduces performance consumption and CPU occupancy, does not affect the performance of the device, and can simultaneously share the audio and video data of the screen, thereby effectively improving the practicality of screen sharing.

[0156] Figure 8 This is a schematic diagram of the structure of a screen sharing device provided in an embodiment of the present application. Figure 8 As shown, the device includes: a sub-process starting module 801, a sub-process data collection module 802, an inter-process data sending module 803 and a transmission module 804.

[0157] The sub-process starting module 801 is used to respond to the sharing instruction triggered by the screen sharing control in the target session page, start the sub-process corresponding to the sharing instruction through the main process corresponding to the online video session, and establish a communication connection between the main process and the sub-process; the sharing instruction instructs to share the screen display page of the target session object to each session object in the online video session;

[0158] The sub-process data collection module 802 is used to collect multimedia data of the screen display page through the sub-process to obtain screen multimedia data, wherein the screen multimedia data includes screen image data and screen audio data corresponding to the screen display page;

[0159] An inter-process data sending module 803, configured to send the screen multimedia data to the main process based on the communication connection between the main process and the sub-process;

[0160] The transmission module 804 is used to transmit the screen multimedia data to the server through the main process, so as to share the screen display page to each session object in the online video session through the server.

[0161] In one possible implementation, the device further includes:

[0162] A startup module, used for displaying a screen sharing startup page through the main process, wherein the screen sharing startup page displays a startup control for triggering the startup of screen sharing;

[0163] The sub-process starting module is used to start the sub-process corresponding to the sharing instruction through the main process in response to the triggering operation of the start control in the screen sharing start page.

[0164] In a possible implementation, the screen sharing startup page further includes a selection control, and the selection control is used to trigger a selection operation for at least one candidate page on the screen;

[0165] The sub-process startup module is used to respond to a selection operation triggered by a selection control and a trigger operation on a startup control, use the page selected based on the selection operation as the screen display page, and start a sub-process for collecting multimedia data for the screen display page through the main process.

[0166] In a possible implementation, an object picture corresponding to the target conversation object is displayed in the target conversation page;

[0167] The device also includes:

[0168] A main process recording module, used for recording an object screen corresponding to a target conversation object through the main process to obtain conversation object data corresponding to the target conversation object;

[0169] The transmission module is used to transmit the screen multimedia data and the session object data to the server through the main process.

[0170] In a possible implementation, the sub-process data collection module is used to:

[0171] Collecting multimedia data from the screen display page through the subprocess;

[0172] For the collected original picture data of the video type, compressing the original picture data to obtain the screen picture data of the screen display page;

[0173] For the collected original audio data of the audio type, the original audio data is used as the screen audio data corresponding to the screen display page.

[0174] In a possible implementation, the conversation object data includes object picture data and object audio data corresponding to the target conversation object;

[0175] The transmission module, when transmitting the screen multimedia data and the session object data to the server through the main process, is specifically used to:

[0176] By means of the main process, the object audio data and the screen audio data are aligned based on the timestamps of the object audio data and the screen audio data, and the aligned object audio data and the screen audio data are compressed, and the compressed object audio data and the screen audio data are sent to the server;

[0177] Through the main process, the object picture data and the screen picture data are aligned based on the timestamps of the object picture data and the screen picture data, and the aligned object picture data and the screen picture data are sent to the server.

[0178] In one possible implementation, the device further includes:

[0179] A business data acquisition module, used to acquire business log data corresponding to the multimedia data collection process of the screen display page through the sub-process, and send the business log data to the main process;

[0180] The service detection module is used to respond to the service detection instruction for the online video session and perform service detection on the multimedia data collection process of the sub-process based on the service log data through the main process.

[0181] In one possible implementation, the device further includes:

[0182] A creation module, used to create a socket communication end of the main process based on the communication address and port of the main process through the main process;

[0183] The sub-process startup module is specifically used to establish a communication connection between the main process and the sub-process:

[0184] A socket communication terminal of the sub-process is created through the sub-process, and a communication connection is established between the socket communication terminal of the main process and the socket communication terminal of the sub-process based on the communication address and port of the main process.

[0185] In one possible implementation, the device further includes:

[0186] A display module, configured to display the screen display page and an end control in the target session page;

[0187] The end module is used to respond to the triggering operation of the end control and send an end notification message to the sub-process through the main process to make the sub-process terminate the multimedia data collection of the screen display page.

[0188] The screen sharing device provided by the present application, when receiving a sharing instruction triggered by a screen sharing control in a target session page, can start a sub-process corresponding to the sharing instruction through a main process, and establish a communication connection between the main process and the sub-process; the sub-process can collect multimedia data for the screen display page to be shared, and obtain screen multimedia data, which includes screen picture data and screen audio data of the screen display page; thereby, the screen and audio of the screen to be shared can be recorded through the sub-process. Moreover, the screen multimedia data is sent to the main process based on the communication connection between processes, and the main process transmits the screen multimedia data to the server, so that the server shares the screen display page to each session object; and since the online video session is implemented by the main process, the screen to be shared and the business of the main process are transmitted based on a session instance; there is no need to take screenshots of the shared screen frame by frame, which reduces performance consumption and CPU occupancy, does not affect the performance of the device, and can simultaneously share the audio and video data of the screen, thereby effectively improving the practicality of screen sharing.

[0189] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0190] Fig. 9 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 As shown, the electronic device includes: a memory, a processor and a computer program stored in the memory, and the processor executes the above computer program to implement the steps of the screen sharing method. Compared with the related art, the following can be achieved:

[0191] The screen sharing method provided by the present application, when receiving a sharing instruction triggered by a screen sharing control in a target session page, can start a subprocess corresponding to the sharing instruction through a main process, and establish a communication connection between the main process and the subprocess; the subprocess can collect multimedia data for the screen display page to be shared, and obtain screen multimedia data, which includes screen picture data and screen audio data of the screen display page; thereby, the screen and audio of the screen to be shared can be recorded through the subprocess. Moreover, the screen multimedia data is sent to the main process based on the communication connection between processes, and the main process transmits the screen multimedia data to the server, so that the server shares the screen display page to each session object; and since the online video session is implemented by the main process, the screen to be shared and the business of the main process are transmitted based on a session instance; there is no need to take screenshots of the shared screen frame by frame, which reduces performance consumption and CPU occupancy, does not affect the performance of the device, and can simultaneously share the audio and video data of the screen, thereby effectively improving the practicality of screen sharing.

[0192] In an alternative embodiment, an electronic device is provided, such as Fig. 9 As shown, Fig. 9 The electronic device 900 shown includes: a processor 901 and a memory 903. The processor 901 and the memory 903 are connected, such as through a bus 902. Optionally, the electronic device 900 may also include a transceiver 904, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 904 is not limited to one, and the structure of the electronic device 900 does not constitute a limitation on the embodiments of the present application.

[0193] Processor 901 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 901 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0194] The bus 902 may include a path to transmit information between the above components. The bus 902 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 902 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0195] The memory 903 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium\other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0196] The memory 903 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 901. The processor 901 is used to execute the computer program stored in the memory 903 to implement the steps shown in the above method embodiment.

[0197] Among them, electronic equipment includes but is not limited to: servers, terminals or cloud computing center equipment, etc.

[0198] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0199] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0200] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0201] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0202] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0203] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A screen sharing method, characterized in that: The method comprises: In response to a sharing instruction triggered by a screen sharing control in a target session page, a subprocess corresponding to the sharing instruction is started through a main process corresponding to the online video session, and a communication connection is established between the main process and the subprocess; the sharing instruction instructs sharing the screen display page of the target session object to each session object in the online video session; Collecting multimedia data of the screen display page through the subprocess to obtain screen multimedia data, wherein the screen multimedia data includes screen picture data and screen audio data corresponding to the screen display page; Based on the communication connection between the main process and the sub-process, sending the screen multimedia data to the main process; The screen multimedia data is transmitted to the server through the main process, so that the screen display page is shared to each session object in the online video session through the server.

2. The method according to claim 1, characterized in that: Before starting the sub-process corresponding to the sharing instruction through the main process corresponding to the online video session, the method further includes: Displaying a screen sharing startup page through the main process, wherein a startup control for triggering the startup of screen sharing is displayed on the screen sharing startup page; The main process corresponding to the online video session starts a subprocess corresponding to the sharing instruction, including: In response to a triggering operation on a start control in the screen sharing start page, a sub-process corresponding to the sharing instruction is started through a main process.

3. The method according to claim 2, characterized in that The screen sharing startup page also includes a selection control, and the selection control is used to trigger a selection operation for at least one candidate page on the screen; The main process corresponding to the online video session starts a subprocess corresponding to the sharing instruction, including: In response to a selection operation triggered by a selection control and a trigger operation on a start control, a page selected based on the selection operation is used as the screen display page, and a subprocess for collecting multimedia data for the screen display page is started through the main process.

4. The method according to claim 1, characterized in that The target conversation page displays an object picture corresponding to the target conversation object; The method further comprises: Recording the object screen corresponding to the target conversation object through the main process to obtain conversation object data corresponding to the target conversation object; The transmitting the screen multimedia data to the server through the main process includes: The screen multimedia data and the session object data are transmitted to the server through the main process.

5. The method according to claim 4, characterized in that The collecting multimedia data of the screen display page through the sub-process to obtain screen multimedia data includes: Collecting multimedia data from the screen display page through the subprocess; For the collected original picture data of the video type, compressing the original picture data to obtain the screen picture data of the screen display page; For the collected original audio data of the audio type, the original audio data is used as the screen audio data corresponding to the screen display page.

6. The method according to claim 5, characterized in that The conversation object data includes object picture data and object audio data corresponding to the target conversation object; The transmitting the screen multimedia data and the session object data to the server through the main process includes: By means of the main process, the object audio data and the screen audio data are aligned based on the timestamps of the object audio data and the screen audio data, and the aligned object audio data and the screen audio data are compressed, and the compressed object audio data and the screen audio data are sent to the server; Through the main process, the object picture data and the screen picture data are aligned based on the timestamps of the object picture data and the screen picture data, and the aligned object picture data and the screen picture data are sent to the server.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining, through the sub-process, business log data corresponding to the multimedia data collection process for the screen display page, and sending the business log data to the main process; In response to the service detection instruction for the online video session, the main process performs service detection on the multimedia data collection process of the sub-process based on the service log data.

8. The method according to claim 1, characterized in that: Before starting the sub-process corresponding to the sharing instruction through the main process corresponding to the online video session, the method further includes: By means of the main process, a socket communication terminal of the main process is created based on the communication address and port of the main process; The establishing of a communication connection between the main process and the sub-process comprises: A socket communication terminal of the sub-process is created through the sub-process, and a communication connection is established between the socket communication terminal of the main process and the socket communication terminal of the sub-process based on the communication address and port of the main process.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: Displaying the screen display page and the end control in the target session page; In response to the triggering operation of the end control, an end notification message is sent to the sub-process through the main process, so that the sub-process terminates the multimedia data collection of the screen display page.

10. A screen sharing device, characterized in that: The device comprises: A sub-process starting module, configured to respond to a sharing instruction triggered by a screen sharing control in a target session page, start a sub-process corresponding to the sharing instruction through a main process corresponding to the online video session, and establish a communication connection between the main process and the sub-process; the sharing instruction instructs sharing the screen display page of the target session object to each session object in the online video session; A sub-process data collection module, used for collecting multimedia data of the screen display page through the sub-process to obtain screen multimedia data, wherein the screen multimedia data includes screen picture data and screen audio data corresponding to the screen display page; An inter-process data sending module, used for sending the screen multimedia data to the main process based on the communication connection between the main process and the sub-process; The transmission module is used to transmit the screen multimedia data to the server through the main process, so as to share the screen display page to each session object in the online video session through the server.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the screen sharing method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the screen sharing method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the screen sharing method according to any one of claims 1 to 9 is implemented.

Images