Audio and video conference implementation method and device, storage medium and computer device

Abstract

The application discloses an audio and video conference implementation method and device, a storage medium and a computer device. The application can receive a target operation event triggered by a user interface acting on an audio and video conference control terminal, the audio and video conference control terminal comprising an interface control layer and a logic processing layer. The application determines feature data of the target operation event and a service identifier of an event processing service corresponding to the target operation event in the interface control layer, and transmits the feature data and the service identifier to the logic processing layer. The application calls the event processing service corresponding to the service identifier to process the event according to the feature data in the logic processing layer, obtains processing result information of the target operation event, and transmits the processing result information to the interface control layer. The application updates the user interface of the audio and video conference control terminal according to the processing result information in the interface control layer, and displays the processing result of the target operation event on the user interface. The application can effectively improve the software development efficiency.

Description

Technical Field

[0001] This application relates to the field of computers, specifically to methods, apparatuses, storage media, and computer equipment for implementing audio and video conferencing. Background Technology

[0002] In the process of software development in the field of audio and video conferencing control, related functions can be implemented based on the native capabilities of the computer device's operating system, or they can be implemented based on code written in a specific programming language. In the process of researching and practicing the existing technology, the inventors of this application have found that when developing related software in the field of audio and video conferencing control based on a programming language, the existing technology is limited by the characteristics of the specific programming language itself, requiring a large amount of code to implement fairly complex logic, resulting in low software development efficiency. Summary of the Invention

[0003] This application provides methods, apparatus, storage media, and computer equipment for implementing audio and video conferencing, which can effectively improve software development efficiency.

[0004] This application provides an audio and video conferencing implementation method, including: The system receives target operation events triggered by the user interface of the audio / video conferencing control terminal, which includes an interface control layer and a logic processing layer. The interface control layer determines the feature data of the target operation event and the service identifier of the event processing service corresponding to the target operation event. The feature data and the service identifier are transmitted to the logic processing layer; Based on the feature data, the logic processing layer calls the event processing service corresponding to the service identifier to process the event and obtain the processing result information of the target operation event. The processing result information is transmitted to the interface control layer, and the interface control layer updates the user interface of the audio and video conferencing control terminal according to the processing result information, so as to display the processing result of the target operation event on the user interface.

[0005] Specifically, this application also provides an audio and video conferencing implementation device, including: The receiving module is used to receive target operation events triggered by the user interface of the audio and video conferencing control terminal, which includes an interface control layer and a logic processing layer. The determination module is used to determine the feature data of the target operation event and the service identifier of the event processing service corresponding to the target operation event in the interface control layer. The transmission module is used to transmit the feature data and the service identifier to the logic processing layer; The calling module is used to call the event processing service corresponding to the service identifier in the logic processing layer according to the feature data to perform event processing and obtain the processing result information of the target operation event; An update module is used to transmit the processing result information to the interface control layer, and the interface control layer updates the user interface of the audio and video conferencing control terminal according to the processing result information, so as to display the processing result of the target operation event on the user interface.

[0006] In some embodiments, the transmission module includes a processing submodule and a transmission submodule, wherein, The processing submodule is used to perform serialization processing on the feature data and the service identifier in the interface control layer to obtain the event data of the target operation event; The transmission submodule is used to transmit the event data of the target operation event to the logic processing layer.

[0007] In some embodiments, the transmission submodule is specifically used for: The event identifier for the target operation event is generated in the interface control layer; Transmit the event data marked by the event identifier to the logic processing layer; At this point, the update module includes a result transmission submodule, a determination submodule, and an update submodule, among which, The result transmission submodule is used to transmit the processing result information marked by the event identifier to the interface control layer; The determination submodule is used to determine the target operation event corresponding to the processing result information based on the event marker in the interface control layer; An update submodule is used to update the user interface of the audio and video conferencing control terminal in the interface control layer according to the processing result information, so as to display the processing result of the target operation event in the user interface.

[0008] In some embodiments, the update submodule is specifically used for: Based on the processing result information, the interface control layer constructs the event state information of the target operation event; Based on the event status information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0009] In some embodiments, the receiving module is specifically used for: Receive event information of target operation events triggered by the user interface of the audio / video conferencing control terminal; At this point, the determination module includes an extraction submodule and a determination submodule, wherein, An extraction submodule is used to extract feature data of the target operation event from the event information in the interface control layer; The determination submodule is used to determine the service identifier of the event handling service corresponding to the target operation event in the interface control layer.

[0010] In some embodiments, the extraction submodule is specifically used for: The interface control layer adds event information of the target operation event to the controller, and the controller extracts the feature data of the target operation event from the event information. At this point, it is determined that the submodule is specifically used for: The interface control layer transmits the feature data of the target operation event to the communication interface. The service identifier of the event processing service corresponding to the target operation event is determined through the communication interface according to the first mapping relationship. The first mapping relationship includes the relationship between multiple preset operation events and preset service identifiers. At this point, the transmission module is specifically used for: The feature data and the service identifier are transmitted to the logic processing layer through the communication interface.

[0011] In some embodiments, the audio and video conferencing apparatus further includes: The message transmission module is used to transmit the meeting change message to the interface control layer when a meeting change message of a preset type is received. The information display module is used to update the user interface of the audio and video conferencing control terminal in the interface control layer according to the meeting change message, so as to display the meeting change information in the user interface.

[0012] In some embodiments, the audio and video conferencing apparatus further includes: The parameter sending module is used to send audio and video parameter information to the operating system where the audio and video conferencing control terminal is located; The playback module is used to receive audio and video stream data transmitted by the operating system based on the audio and video parameter information in the interface control layer, and to play audio and video based on the audio and video stream data in the user interface.

[0013] In some embodiments, the playback module is specifically used for: The operating system generates an audio and video transmission identifier based on the audio and video parameter information, sends the audio and video transmission identifier to the interface control layer, and receives audio and video stream data returned by the server based on a data request message. The data request message is a message sent by the operation to the server based on the audio and video parameter information. An audio and video transmission channel corresponding to the audio and video transmission identifier is constructed in the interface control layer, and audio and video stream data transmitted by the operating system through the audio and video transmission channel is received. The user interface plays audio and video based on the audio and video stream data.

[0014] In some embodiments, the user interface includes a responsive interface and a control interface, and the audio / video conferencing device further includes: The response page display module is used to display a response interface for the conference hardware device on the audio and video conferencing control terminal. The response interface includes a conference initiation control corresponding to the conference hardware device. The response interface is the interface displayed by the terminal in response to the conference information of the conference hardware device. The control page display module is used to respond to the operation of the meeting initiation control and display the control interface of the target audio and video conference initiated by the audio and video conference control terminal. The control interface includes meeting control controls for the target audio and video conference and device control controls for the meeting hardware device. The meeting control controls are used to change the meeting status of the target audio and video conference, and the device control controls are used to change the device status of the meeting hardware device.

[0015] In some embodiments, the conference control controls include member audio / video control controls and member management controls, and the audio / video conferencing implementation device further includes: The output change module is used to respond to the operation of the member audio and video control control to trigger a change in the audio and video output state of the conference equipment used by the conference members in the target audio and video conference; The state change module is used to respond to operations on the member management control and trigger a change in the meeting state of the meeting members in the target audio and video conference.

[0016] In some embodiments, the device control controls include device audio / video control controls and device control exit controls, and the audio / video conferencing device further includes: The hardware modification module is used to trigger a change in the audio and video output state of the conference hardware device in response to an operation on the device's audio and video control controls. The control change module is used to respond to the operation of the device control exit control and release the control state of the audio and video conferencing control terminal on the conferencing hardware device.

[0017] Accordingly, this application also provides a storage medium storing a computer program adapted for loading by a processor to execute any of the audio / video conferencing implementation methods provided in this application.

[0018] Accordingly, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the audio and video conferencing methods provided in this application.

[0019] This application can receive target operation events triggered by the user interface of an audio / video conferencing control terminal. The audio / video conferencing control terminal includes an interface control layer and a logic processing layer. The interface control layer determines the characteristic data of the target operation event and the service identifier of the event processing service corresponding to the target operation event. The characteristic data and the service identifier are transmitted to the logic processing layer. Based on the characteristic data, the logic processing layer calls the event processing service corresponding to the service identifier to process the event and obtain the processing result information of the target operation event. The processing result information is transmitted to the interface control layer. Based on the processing result information, the interface control layer updates the user interface of the audio / video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0020] This application enables the control of audio and video conferencing via an audio and video conferencing control terminal comprising an interface control layer and a logic processing layer. Specifically, during the control process, the interface control layer determines the target operation event triggered by the user's operation on the user interface, the characteristic data and service identifier of the target operation event, and transmits the characteristic data and service identifier to the logic processing layer, which then performs the invocation and other processes. This application constructs the audio and video conferencing control terminal in a layered manner, with the logic processing layer and the interface control layer each implementing different functions. Different programming languages ​​are used for development based on the characteristics of the functions implemented by each layer. Compared with the existing technology that uses a single programming language to develop the entire software, this application can significantly improve software development efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the audio and video conferencing system implemented in this application; Figure 2 This is a flowchart illustrating the audio and video conferencing implementation method of this application; Figure 3 This is a user interface diagram illustrating the audio and video conferencing implementation method of this application; Figure 4 This is another user interface diagram illustrating the audio and video conferencing implementation method of this application; Figure 5 This is another user interface diagram illustrating the audio and video conferencing implementation method of this application; Figure 6 This is another user interface diagram illustrating the audio and video conferencing implementation method of this application; Figure 7 This is another user interface diagram illustrating the audio and video conferencing implementation method of this application; Figure 8 This is another flowchart illustrating the audio and video conferencing implementation method of this application; Figure 9 This is a partial architectural diagram of the audio and video conferencing implementation method of this application; Figure 10 This is another part of the technical process diagram of the audio and video conferencing implementation method of this application; Figure 11 This is a partial technical flow diagram of the audio and video conferencing implementation method of this application; Figure 12 This is another part of the technical process diagram of the audio and video conferencing implementation method of this application; Figure 13 This is another part of the technical process diagram of the audio and video conferencing implementation method of this application; Figure 14 This is another part of the technical process diagram of the audio and video conferencing implementation method of this application; Figure 15 This is another part of the technical process diagram of the audio and video conferencing implementation method of this application; Figure 16 This is a schematic diagram of the audio and video conferencing implementation device of this application; Figure 17 This is a schematic diagram of the computer device described in this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described in this application are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The audio and video conferencing implementation method of this application can be integrated into an audio and video conferencing implementation device, which can be integrated into an audio and video conferencing implementation system. The audio and video conferencing implementation system can include one or more computer devices, which can include at least one of terminals, servers, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart conferencing device, smart screen, etc., but is not limited thereto. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0025] See Figure 1 , Figure 1 The present application provides a schematic diagram of the audio and video conferencing implementation method. The audio and video conferencing implementation method can be integrated into a terminal and a conferencing hardware device. The terminal may include an audio and video conferencing control terminal, which can display a user interface. Users can operate the conferencing hardware device through the user interface of the terminal, such as controlling the conferencing hardware device to turn on the camera, etc.

[0026] It should be noted that, Figure 1 The schematic diagram of the audio and video conferencing system shown is merely an example. The audio and video conferencing system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of audio and video conferencing devices and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0027] The following sections provide detailed explanations. In this embodiment, the audio and video conferencing implementation method will be described in detail. This method can be integrated into a terminal, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating the audio and video conferencing implementation method provided in this application embodiment. The audio and video conferencing implementation method may include: 101. Receive target operation events triggered by the user interface of the audio / video conferencing control terminal. The audio / video conferencing control terminal includes an interface control layer and a logic processing layer.

[0028] The audio and video conferencing control terminal may include an application that enables online audio and video conferencing on computer devices. Users can initiate or participate in audio and video conferencing through the audio and video conferencing control terminal. The user interface may include the interface that the audio and video conferencing control terminal can display on the terminal. Users can operate on the user interface to achieve the corresponding functions.

[0029] The target operation events can include events triggered by user operations on the user interface during the target audio and video conference process. For example, when a user clicks the "Initiate Meeting" control on the user interface, the corresponding target operation event triggered could be "Create a meeting initiated by the user". Similarly, target operation events could also include "Invite meeting members", "Mute all meeting members", "Upload file A in the meeting", "End the meeting", etc.

[0030] The audio and video conferencing control terminal of this application may include an interface control layer and a logic processing layer. This application can control the user interface through the interface control layer. For example, the interface control layer can determine the target operation event triggered by the user's operation on the user interface, update the displayed content of the user interface, etc. This application can process the target operation event through the logic processing layer. Specifically, the logic processing layer can process network-related content, such as sending a meeting creation request to the server or receiving data sent by the server, etc. The logic processing layer can also perform related logical calculations and interface calls, etc.

[0031] For example, the interface control layer can display files uploaded by meeting members and change the meeting members' participation status on the user interface. The logic processing layer can receive text data sent by the server and call the voice interface to change the terminal's voice volume, and so on.

[0032] The interface control layer and logic processing layer of this application can be constructed using different programming languages. Specifically, they can be flexibly selected based on the characteristics of existing programming languages ​​and the functions implemented by the interface control layer and logic processing layer, respectively. For example, the interface control layer can use the Dart language, and the logic processing layer can use the C++ language.

[0033] For example, if user A invites C to join meeting 1 by triggering a control in the user interface, the interface control layer of the audio and video conferencing terminal can receive the target operation event 1 (inviting C to join meeting 1) triggered by user A's operation in the user interface.

[0034] 102. In the interface control layer, determine the characteristic data of the target operation event and the service identifier of the event handling service corresponding to the target operation event.

[0035] The feature data can include data that characterizes the features of the target operation event. The data type and quantity of the feature data can be flexibly set according to the actual scenario. For example, when the target operation event is to unmute all users, the feature data can include the Boolean type false. Or, when the target operation event is to upload file A, the feature data can include file A, and so on.

[0036] Event handling services can include services called when the logic processing layer processes target operation events. Event handling services can be interfaces, methods, classes, functions, etc., and different event handling services are different. The service identifier is information that uniquely identifies the event handling service. The service identifier can be in the form of characters, text, etc., and can be flexibly handled in the application process.

[0037] Specifically, the terminal can obtain the feature data corresponding to the target operation event through a preset event data mapping relationship. The terminal can pre-set an event service mapping relationship, which can include the service identifier of the event processing service corresponding to different operation events. Then, the interface control layer can determine the service identifier of the event processing service corresponding to the target operation event from the event service mapping relationship.

[0038] For example, the characteristic data 1 of the target operation event 1 can be determined in the interface control layer, and the service identifier P of the event handling service corresponding to the target operation event 1 can be determined.

[0039] In some embodiments, the step "receiving a target operation event triggered by the user interface of the audio / video conferencing control terminal" may include: Receive event information of target operation events triggered by the user interface of the audio / video conferencing control terminal; The step "determine the characteristic data of the target operation event and the service identifier of the event handling service corresponding to the target operation event in the interface control layer" may include: In the interface control layer, feature data of the target operation event is extracted from the event information; In the interface control layer, determine the service identifier of the event handling service corresponding to the target operation event.

[0040] The event information may include information recorded in the form of data about the target operation event. The interface control layer can receive the event information of the target operation event, extract the feature data of the target operation event from the event information, and then determine the service identifier of the event processing service corresponding to the target operation event.

[0041] In some embodiments, the step "extracting feature data of the target operation event from the event information in the interface control layer" may include: Add event information of the target operation event to the controller in the interface control layer, and extract the feature data of the target operation event from the event information through the controller; The step "determine the service identifier of the event handling service corresponding to the target operation event in the interface control layer" may include: The interface control layer transmits the characteristic data of the target operation event to the communication interface. The service identifier of the event processing service corresponding to the target operation event is determined through the communication interface according to the first mapping relationship. The first mapping relationship includes the relationship between multiple preset operation events and preset service identifiers. The step "transmitting feature data and service identifier to the logic processing layer" may include: Feature data and service identifiers are transmitted to the logic processing layer through the communication interface.

[0042] In some embodiments, the interface control layer can be further subdivided into multiple modules. The specific form of the modules can be components, interfaces, classes, instances, functions, etc. Each module can implement different functions. For example, a module can consist of a controller, a communication interface, etc. Specifically, in the process of determining feature data and service identifiers, the event information of the target operation event can be added to the controller. The controller extracts the feature data of the target operation event from the event information and then sends the feature data to the communication interface of the interface control layer. The communication interface determines the service identifier of the event processing service of the target operation event pair. Then, the communication interface can transmit the obtained feature data and service identifier to the logic processing layer.

[0043] 103. Transmit feature data and service identifiers to the logic processing layer.

[0044] This application implements layered processing for the audio and video conferencing control terminal. The interface control layer and the logic processing layer process data and logic relatively independently. However, if the target operation event needs to be processed by the logic processing layer, it is necessary to transmit feature data and service identifiers to the logic processing layer.

[0045] For example, the characteristic data 1 and service identifier P of the target operation event 1 are transmitted from the interface control layer to the logic processing layer.

[0046] In some embodiments, the step "transmitting feature data and service identifier to the logic processing layer" may include: In the interface control layer, the feature data and service identifier are serialized to obtain the event data of the target operation event; Transmit the event data of the target operation event to the logic processing layer.

[0047] Before data transmission between different layers, feature data and service identifiers can be processed to obtain event data that is easy to transmit. Specifically, serialization processing can be based on a certain data structure, such as proto or JSON. Then, the event data of the target operation event can be transmitted to the logic processing layer. Correspondingly, after receiving the event data, the logic processing layer can parse the event data to obtain feature data and service identifiers.

[0048] For example, feature data 1 and service identifier P can be serialized based on the relevant rules of proto to obtain event data A of target operation event 1, and then the event data A can be transmitted to the logic processing layer.

[0049] 104. In the logic processing layer, based on the feature data, the event processing service corresponding to the service identifier is called to process the event and obtain the processing result information of the target operation event.

[0050] The processing result information may include the processing result of the target operation event obtained after processing the event through the event processing service. For example, the processing result information may include successful processing, or it may include processing failure due to network reasons, and so on.

[0051] The logic processing layer can include multiple services that implement specific functions. These services can be interfaces, classes, methods, functions, etc., and can be flexibly handled in actual applications. In the logic processing layer, based on the received service identifier, the corresponding event handling service (i.e., the event handling service for the target operation event) can be determined. Then, based on the feature data, the event handling service can be invoked to process the target operation event and obtain the processing result information. There are several ways to invoke the event handling service based on feature data. For example, the invocation method of the event handling service can be determined based on the feature data, and then the event handling service can be invoked using this method. Alternatively, the feature data can be input into the event handling service to invoke it.

[0052] For example, in the logic processing layer, based on feature data 1, the event processing service corresponding to service identifier P can be called to process the event and obtain the processing result information of target operation event 1.

[0053] 105. Transmit the processing result information to the interface control layer. Based on the processing result information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0054] After the logic processing layer receives the processing result information, it can send the processing result information to the interface control layer. Specifically, the processing result information can be sent through the communication interface on the logic processing layer.

[0055] The interface control layer can update the user interface based on the processing result information. The user interface can display the results of the buttons triggered by the user. For example, if the user uploads file A in the user interface, after receiving the successful upload result information, the interface control layer can display "Upload successful" to prompt the user that file A has been successfully uploaded.

[0056] In some embodiments, the step of "transmitting event data of the target operation event to the logic processing layer" may include: Generate event identifiers for the target operation events in the interface control layer; Transmit event data marked with event identifiers to the logic processing layer; Correspondingly, the step "transmitting processing result information to the interface control layer, and updating the user interface of the audio / video conferencing control terminal based on the processing result information in the interface control layer, so as to display the processing result of the target operation event on the user interface" may include: Transmit the processing result information marked by the event identifier to the interface control layer; In the interface control layer, the target operation event corresponding to the processing result information is determined based on the event marker; Based on the processing result information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0057] The target operation event of this application requires the cooperation of the interface control layer and the logic processing layer. Since they are two relatively independent layers, an event identifier can be set for the target operation event to facilitate data identification and processing. Specifically, the event identifier of the target operation event can be generated in the interface control layer. The feature data, service identifier, and event data obtained by serializing the feature data and service identifier of the target operation event can all be marked by the event identifier.

[0058] The interface control layer can send event data marked by event identifiers to the logic processing layer. After the logic processing layer obtains the processing result information, it can mark the processing result information by event identifiers and send the processing result information marked by event identifiers to the interface control layer. The interface control layer can determine the target operation event corresponding to the processing result information based on the event identifiers, and then update the user interface accordingly based on the processing result information of the target operation event.

[0059] For example, an event identifier VV for the target operation event 1 can be generated in the interface control layer, and then the event data marked by the event identifier VV can be transmitted to the logic processing layer. After the logic processing layer obtains the processing result information, it can send the processing result information marked by the event identifier VV back to the interface control layer. The interface control layer determines that the processing result information is the result of the target operation event 1 based on the event identifier VV, and then updates the user interface based on the processing result information.

[0060] In some embodiments, the step "updating the user interface of the audio / video conferencing control terminal in the interface control layer according to the processing result information, so as to display the processing result of the target operation event in the user interface" may include: Based on the processing result information, the interface control layer constructs the event state information of the target operation event; Based on event status information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing results of the target operation event on the user interface.

[0061] The event status information can include information representing the current state of the target operation event. The communication interface of the interface control layer can receive the processing result information. Then, the controller can identify and process the processing result information to construct the event status information of the target operation event. Then, the user interface component can determine the parts of the user interface that need to be updated based on the event status information and update the user interface. The updated user interface can display the processing result of the target operation event.

[0062] In some embodiments, the audio and video conferencing implementation method may further include: When a meeting change message of a preset type is received, the meeting change message is transmitted to the interface control layer; Based on the meeting change message, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the meeting change information on the user interface.

[0063] In some scenarios, the audio / video conferencing control terminal receives messages from the server regarding the audio / video conferencing. These messages are sent by the server, rather than being actively requested by the audio / video conferencing control terminal. Some of these messages need to be communicated to the user, which means that the user interface needs to be updated. Specifically, meeting change messages can include messages that indicate changes to the current state of the ongoing audio / video conferencing. For example, meeting change messages can indicate that the meeting has ended, the meeting has been paused, meeting members have been muted, the current audio meeting has been changed to a video meeting, and so on.

[0064] After receiving a meeting change message from the server, the logic processing layer can identify the type of the meeting change message. When the meeting change message belongs to a preset type, that is, when the meeting change message needs to be notified to the user, the meeting change message can be transmitted to the interface control layer, so that the interface control layer updates the user interface of the audio and video conferencing control terminal according to the meeting change message.

[0065] For example, if a meeting change notification 1 is received from the server and it is determined that the meeting change message is of type 1, then the meeting change message is sent to the interface control layer. The interface control layer updates the user interface according to the meeting change message, and the updated user interface displays "At the invitation of the host, please start a video conference".

[0066] In some embodiments, the audio and video conferencing implementation method may further include: Send audio and video parameter information to the operating system of the audio and video conferencing control terminal; The interface control layer receives audio and video stream data transmitted by the operating system based on audio and video parameter information, and the user interface plays audio and video based on the audio and video stream data.

[0067] The audio / video conferencing control terminal can also play audio and video. This playback can be real-time audio or video from the online meeting. Specifically, the control terminal can send audio and video parameter information to the operating system of its host computer. This information can include parameters such as resolution, frame rate, and bitrate of the audio and video to be played. The control terminal then receives audio and video stream data from the operating system that matches the audio and video parameter information and plays the audio and video based on this data.

[0068] In some embodiments, the step of "receiving audio and video stream data transmitted by the operating system based on audio and video parameter information at the interface control layer, and playing audio and video based on the audio and video stream data at the user interface" may include: The operating system generates an audio and video transmission identifier based on the audio and video parameter information, sends the audio and video transmission identifier to the interface control layer, and receives the audio and video stream data returned by the server based on the data request message. The data request message is a message sent by the operating system to the server based on the audio and video parameter information. In the interface control layer, an audio and video transmission channel corresponding to the audio and video transmission identifier is constructed. The audio and video stream data transmitted by the operating system through the audio and video transmission channel is received, and the audio and video is played in the user interface based on the audio and video stream data.

[0069] The audio and video transmission identifier can include information that uniquely identifies audio and video transmission events, and the audio and video transmission channel can include the content used to transmit audio and video data between the audio and video conferencing control terminal and the operating system.

[0070] For example, the audio / video conferencing control terminal can determine the audio / video parameter information 1 of the audio / video data to be played based on user operation, and then send the audio / video parameter information 1 to the operating system. The operating system can generate a data request message based on the received audio / video parameter message 1, and send the data request message to the server. The operating system can receive the audio / video data returned by the server based on the data request message, and send the audio / video data to the audio / video conferencing control terminal through the audio / video transmission channel. The audio / video conferencing control terminal can then play the audio / video based on the audio / video data.

[0071] In some embodiments, the user interface includes a responsive interface and a control interface, and the audio and video conferencing implementation method may further include: The audio and video conferencing control terminal displays a response interface for the conferencing hardware device. The response interface includes a meeting initiation control corresponding to the conferencing hardware device. The response interface is the interface displayed by the terminal in response to the meeting information from the conferencing hardware device. In response to the operation of the meeting initiation control, the control interface of the target audio and video conference initiated by the audio and video conferencing control terminal is displayed. The control interface includes meeting control controls for the target audio and video conference and device control controls for the meeting hardware devices. The meeting control controls are used to change the meeting status of the target audio and video conference, and the device control controls are used to change the device status of the meeting hardware devices.

[0072] The controls in this application can be buttons, selection boxes, sliders, edit boxes, etc. Operations on the controls can include clicking (such as tapping, double-clicking, single-clicking), sliding, long-pressing, etc. The response in this application can be used to represent the conditions or states on which the operation is performed. When the conditions or states on which the operation is performed are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0073] For example, see Figure 3 The page for the conference hardware equipment can be like... Figure 3 The audio and video conferencing control terminal can scan Figure 3 The QR code in the image will display something like this. Figure 4The response page 01 may include an audio conferencing button 011 (i.e., an audio initiation control) and a video conferencing button 012 (i.e., a video conferencing initiation control). Users can click the video conferencing button 012 to display the screen of the target video conference they initiated on the conferencing hardware device. The audio and video conferencing control terminal can display the control page 02 of the target video conference. The control page 02 may include a conferencing hardware device management button 021 (i.e., a conferencing control control) for the target video conference and a conferencing hardware device management button 022 (i.e., a device control control) for managing participating members and devices.

[0074] In some embodiments, the conference control controls include member audio / video control controls and member management controls, and the audio / video conferencing implementation method may further include: In response to operations on member audio / video control controls, trigger changes to the audio / video output state of the conferencing devices used by the members in the target audio / video conference; In response to an action on the member management control, trigger a change in the meeting state of the meeting members in the target audio / video conference.

[0075] Member audio / video control controls and member management controls can be displayed on a new independent page or on the control page. The specific design can be flexibly implemented in actual applications without any restrictions.

[0076] For example, trigger Figure 4 After clicking button 021 on the management conference hardware device, the following will be displayed: Figure 5 The meeting control page 03 shown can display information about members already in the meeting and members not yet in the meeting. The meeting control page 03 also displays member meeting audio and video control controls (the "Turn Off All Video" button 031) and member management controls (the "Add Participant" button 032 and the "Add Meeting Device" button 033). Triggering the "Turn Off All Video" button 031 will turn off the video of all participants and devices. Triggering the "Add Participant" button 032 will invite new people to join the meeting. Triggering the "Add Meeting Device" button 033 will allow you to invite new devices to join the meeting.

[0077] In some embodiments, the device control controls include device audio and video control controls and device control exit controls. The audio and video conferencing implementation method may further include: In response to operations on the device's audio / video control controls, trigger changes to the audio / video output state of the conference hardware device; In response to an operation on the device control exit control, the audio / video conferencing control terminal releases control of the conferencing hardware device.

[0078] The device audio and video control controls and the device operation exit control can be displayed on a new independent page or on the control page. The specific design can be flexibly implemented in actual application without any restrictions.

[0079] For example, see Figure 6 The control page, Figure 6 The control page 02 includes device audio and video control controls (mute button 0021 and camera off button 0022) and device control exit control (exit device control button 0023). Triggering the mute button 0021 can mute the conference hardware device, triggering the camera off button 0022 can turn off the camera of the conference hardware device, and triggering the exit device control button 0023 can end the audio and video conference control terminal's control of the conference hardware device.

[0080] In addition, the audio and video conferencing control terminal can receive meeting status change information transmitted from the server and display it on the user interface, such as... Figure 7 The control page 02 can display the message "The host invites the current device to turn on video. The microphone will also be turned on at the same time after the video is turned on," as well as a video start button and a cancel button.

[0081] This application enables the control of audio and video conferencing via an audio and video conferencing control terminal comprising an interface control layer and a logic processing layer. Specifically, during the control process, the interface control layer determines the target operation event triggered by the user's operation on the user interface, the characteristic data and service identifier of the target operation event, and transmits the characteristic data and service identifier to the logic processing layer, which then performs the invocation and other processes. This application constructs the audio and video conferencing control terminal in a layered manner, with the logic processing layer and the interface control layer each implementing different functions. Different programming languages ​​are used for development based on the characteristics of the functions implemented by each layer. Compared with the existing technology that uses a single programming language to develop the entire software, this application can significantly improve software development efficiency.

[0082] Based on the methods described in the above embodiments, the following examples will provide further detailed explanations.

[0083] See Figure 8 , Figure 8 This is another flowchart of the audio and video conferencing implementation method of this application, which may include: 201. The terminal receives event information of the target operation event triggered by the user interface of the audio and video conferencing control terminal. The audio and video conferencing control terminal includes an interface control layer and a logic processing layer.

[0084] 202. The terminal extracts the feature data of the target operation event from the event information at the interface control layer.

[0085] 203. The terminal determines the service identifier of the event handling service corresponding to the target operation event in the interface control layer.

[0086] 204. The terminal generates an event identifier for the target operation event in the interface control layer, performs serialization processing on the feature data and service identifier, and obtains the event data of the target operation event.

[0087] 205. The terminal transmits event data marked by event identifiers to the logic processing layer.

[0088] 206. The terminal processes the event data at the logic processing layer to obtain feature data and service identifiers, and calls the event processing service corresponding to the service identifier based on the feature data to process the event and obtain the processing result information of the target operation event.

[0089] 207. The terminal transmits the processing result information marked by the event identifier to the interface control layer. Based on the processing result information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0090] See Figure 9 , Figure 9 This is a technical architecture diagram of the audio / video conferencing control terminal. Specifically, this technical architecture diagram can be divided according to the programming language used in the development of the audio / video conferencing control terminal, for example... Figure 9 The system can be divided into the Dart layer and the Cpp layer. The Dart layer and the Cpp layer can communicate based on Remote Procedure Call (RPC) technology. The Dart layer and the Cpp layer can use the protobuf data description language to serialize and convert data, facilitating data transmission between them. An observer-consumer model can be built between the Dart layer and the Cpp layer. The specific implementation can be achieved by the cooperation of components deployed in each layer (such as the observer component in the browser kernel module of the Cpp layer, and the observer component in the Dart layer).

[0091] The Cpp layer can handle network requests from the audio and video conferencing control end, as well as complex logical calculations, providing a stable and efficient service for the Dart layer's user interface. The Dart layer can determine the service interfaces to be called in the Cpp layer, and implement the display and interaction of the user interface.

[0092] The Dart layer utilizes user interface components (UI components), business logic components (Bloc), the protobuf data description language, remote procedure call technology, and the observer pattern. Specifically, business logic components can include listeners, constructors, events, and states. User interface components can be used to composite the pages displayed on the audio / video conferencing control terminal.

[0093] In the Dart layer, see Figure 10 , Figure 10 The diagram illustrates the relationships between some components within the Dart layer. Specifically, user interface components can add events to business logic components. After processing these events, the business logic components can return status to the UI. They can also request data from data components that store the data, based on the event information. The data components, in turn, return the requested data to the business logic components. Communication between the business logic and data components can be asynchronous, with requests and responses occurring asynchronously. In this process, Bloc separates data from the UI, allowing data to drive UI changes and facilitating data communication between multiple pages, using state to drive UI modifications.

[0094] The network and logic processing layer (i.e., the Cpp layer) can include browser kernel modules, engine modules, utility class providing modules such as Utils, etc. The browser kernel can be the open-source Chromium (a browser kernel). Chromium can include service components, observer components, HTTP-based transport components, encryption and decryption components, basic module components, etc. The engine module (FlutterEngine) can include communication ports, independent running space, communication channels, runtime environment, vector graphics processing function library, etc.

[0095] The flowchart for implementing mute in the audio / video conferencing control terminal of this application can be found in [link to application]. Figure 11Specifically, when a user clicks a mute control (such as a mute button) on the user page of the audio / video conferencing control terminal, the audio / video conferencing control terminal obtains the global meeting control logic component (HardwareMeetingControllerBloc), adds a mute event (ControlMemberAudioAndVideoEvent) in the meeting control logic component, and the meeting control logic component can listen to this mute event (through the mapEventToState method), extract the data in the mute event (such as the boolean data true indicating mute), construct a data packet PVControlReq, and then send the data packet to the remote procedure call service. The mute is implemented in the Cpp layer through the remote procedure call service, and the terminal asynchronously waits for the call result returned by the Cpp layer from the remote procedure call service, inputs the call result into the stream data, determines the state corresponding to the call result, and the user interface listens to the stream data, can receive the state of the stream data transmission, and update the user interface according to the state.

[0096] The flowchart illustrating the communication between different layers of the audio / video conferencing control terminal in this application can be found in [reference]. Figure 12 The Dart layer can create a HardwareRpcServiceApi instance, with the constructor being WeRpcClient. Then, it asynchronously calls the corresponding method through HardwareRpcServiceApi, sending the data in the method to the proxy class RPC client. The proxy class's invoke method packages the data to be called and the service name into a new data packet pbbuffer. The proxy class can generate a unique requestId and register a callback to the global callBackCenter. Then, dart2cpp sends the pb buffer to the C++ layer. After receiving the data packet, the C++ layer finds the corresponding service through the service name and method name, implements the corresponding method call through static proxy, and returns the result and callbackId to the Dart layer. The Dart layer finds the corresponding callback based on the requestId and executes the corresponding callback operation.

[0097] The flowchart for receiving meeting change notifications by the audio / video conferencing control terminal in this application can be found in [reference]. Figure 13Register the message types to be listened to in CppObserverCenter and return a stream of data. The Dart layer stores the data stream in a map (data structure) according to the type. The Cpp layer initializes the observer, implements a message notification observer-consumer model, registers consumer events, and sends all data to the Dart layer through dart2cpp. The Dart layer's caller receives the data stream and processes the relevant audio and video business. The Dart layer unregisters the observer when its lifecycle ends to avoid memory leaks.

[0098] The flowchart for the audio / video conferencing user interface state switching in this application can be found in [reference needed]. Figure 14 The UI listens to the bloc data stream through BlocBuilder. After Bloc processes the data, it constructs different states based on the status codes in the background and returns the data to the UI. The UI implements different UI logic based on the different states.

[0099] The flowchart for the audio / video conferencing user interface state switching in this application can be found in [reference needed]. Figure 15 A Texture widget is created, and some data (initialized data, such as resolution and clarity) is sent to Android / iOS via a channel. Android / iOS generates a unique video stream ID and returns this data to Flutter. After obtaining this video stream ID, an EventChannel data transmission channel is established. The native code converts the video into a PixelBuffer (a data format) data stream and sends the video stream data through the EventChannel. After Flutter receives the video stream data, it starts rendering, thus enabling audio and video playback in the UI.

[0100] This application enables the control of audio and video conferencing via an audio and video conferencing control terminal comprising an interface control layer and a logic processing layer. Specifically, during the control process, the interface control layer determines the target operation event triggered by the user's operation on the user interface, the characteristic data and service identifier of the target operation event, and transmits the characteristic data and service identifier to the logic processing layer, which then performs the invocation and other processes. This application constructs the audio and video conferencing control terminal in a layered manner, with the logic processing layer and the interface control layer each implementing different functions. Different programming languages ​​are used for development based on the characteristics of the functions implemented by each layer. Compared with the existing technology that uses a single programming language to develop the entire software, this application can significantly improve software development efficiency.

[0101] This application also provides an apparatus based on the above-described audio and video conferencing implementation method. The meanings of the terms used are the same as in the above-described audio and video conferencing implementation method, and specific implementation details can be found in the descriptions in the method embodiments.

[0102] See Figure 16 , Figure 16 This is a schematic diagram of the audio and video conferencing implementation device provided in this application. The device includes a receiving module 301, a determining module 302, a transmitting module 303, a calling module 304, and an updating module 305. The receiving module 301 is used to receive target operation events triggered by the user interface of the audio and video conferencing control terminal. The audio and video conferencing control terminal includes an interface control layer and a logic processing layer. The determination module 302 is used to determine the feature data of the target operation event and the service identifier of the event processing service corresponding to the target operation event in the interface control layer. Transmission module 303 is used to transmit feature data and service identifiers to the logic processing layer; Module 304 is invoked to call the event processing service corresponding to the service identifier in the logic processing layer based on the feature data to process the event and obtain the processing result information of the target operation event. The update module 305 is used to transmit processing result information to the interface control layer. Based on the processing result information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0103] In some embodiments, the transmission module includes a processing submodule and a transmission submodule, wherein, The processing submodule is used to serialize feature data and service identifiers in the interface control layer to obtain event data of the target operation event; The transmission submodule is used to transmit event data of the target operation event to the logic processing layer.

[0104] In some embodiments, the transmission submodule is specifically used for: Generate event identifiers for the target operation events in the interface control layer; Transmit event data marked with event identifiers to the logic processing layer; At this point, the update module includes a result transmission submodule, a determination submodule, and an update submodule, among which, The result transmission submodule is used to transmit the processing result information marked by the event identifier to the interface control layer; The determination submodule is used to determine the target operation event corresponding to the processing result information based on the event marker in the interface control layer; The update submodule is used to update the user interface of the audio and video conferencing control terminal based on the processing result information in the interface control layer, so as to display the processing result of the target operation event on the user interface.

[0105] In some embodiments, the update submodule is specifically used for: Based on the processing result information, the interface control layer constructs the event state information of the target operation event; Based on event status information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing results of the target operation event on the user interface.

[0106] In some embodiments, the receiving module is specifically used for: Receive event information of target operation events triggered by the user interface of the audio / video conferencing control terminal; At this point, the determination module includes an extraction submodule and a determination submodule, wherein, The extraction submodule is used to extract feature data of the target operation event from the event information in the interface control layer; The determination submodule is used to determine the service identifier of the event handling service corresponding to the target operation event in the interface control layer.

[0107] In some embodiments, the extraction submodule is specifically used for: Add event information of the target operation event to the controller in the interface control layer, and extract the feature data of the target operation event from the event information through the controller; At this point, it is determined that the submodule is specifically used for: The interface control layer transmits the characteristic data of the target operation event to the communication interface. The service identifier of the event processing service corresponding to the target operation event is determined through the communication interface according to the first mapping relationship. The first mapping relationship includes the relationship between multiple preset operation events and preset service identifiers. At this point, the transmission module is specifically used for: Feature data and service identifiers are transmitted to the logic processing layer through the communication interface.

[0108] In some embodiments, the audio and video conferencing apparatus further includes: The message transmission module is used to transmit meeting change messages to the interface control layer when a meeting change message of a preset type is received. The information display module is used to update the user interface of the audio and video conferencing control terminal based on the meeting change message in the interface control layer, so as to display the meeting change information on the user interface.

[0109] In some embodiments, the audio and video conferencing apparatus further includes: The parameter sending module is used to send audio and video parameter information to the operating system where the audio and video conferencing control terminal is located; The playback module is used to receive audio and video stream data transmitted by the operating system based on audio and video parameter information at the interface control layer, and to play audio and video based on the audio and video stream data at the user interface.

[0110] In some embodiments, the playback module is specifically used for: The operating system generates an audio and video transmission identifier based on the audio and video parameter information, sends the audio and video transmission identifier to the interface control layer, and receives the audio and video stream data returned by the server based on the data request message. The data request message is a message sent by the operation to the server based on the audio and video parameter information. In the interface control layer, an audio and video transmission channel corresponding to the audio and video transmission identifier is constructed. The audio and video stream data transmitted by the operating system through the audio and video transmission channel is received, and the audio and video is played in the user interface based on the audio and video stream data.

[0111] In some embodiments, the user interface includes a responsive interface and a control interface, and the audio / video conferencing device further includes: The response page display module is used to display the response interface for the conference hardware device on the audio and video conferencing control terminal. The response interface includes the conference initiation control corresponding to the conference hardware device. The response interface is the interface displayed by the terminal in response to the conference information of the conference hardware device. The control page display module is used to respond to operations on the meeting initiation control and display the control interface of the target audio and video conference initiated by the audio and video conference control terminal. The control interface includes meeting control controls for the target audio and video conference and device control controls for the meeting hardware devices. The meeting control controls are used to change the meeting status of the target audio and video conference, and the device control controls are used to change the device status of the meeting hardware devices.

[0112] In some embodiments, the conference control controls include member audio / video control controls and member management controls, and the audio / video conferencing implementation device further includes: The output change module is used to respond to operations on the member audio and video control controls and trigger changes to the audio and video output state of the conference equipment used by the conference members in the target audio and video conference. The state change module is used to respond to operations on the member management control and trigger changes to the meeting state of the meeting members in the target audio and video conference.

[0113] In some embodiments, the device control controls include device audio / video control controls and device control exit controls; the audio / video conferencing apparatus further includes: The hardware modification module is used to respond to operations on the device's audio and video control controls and trigger changes to the audio and video output state of the conference hardware device. The control change module is used to respond to operations on the device control exit control, thereby releasing the audio and video conferencing control terminal from the control status of the conferencing hardware device.

[0114] In this application, the receiving module 301 can receive target operation events triggered by the user interface of the audio / video conferencing control terminal. The audio / video conferencing control terminal includes an interface control layer and a logic processing layer. The determining module 302 determines the feature data of the target operation event and the service identifier of the event processing service corresponding to the target operation event in the interface control layer. Then, the transmitting module 303 transmits the feature data and service identifier to the logic processing layer. Next, the calling module 304 calls the event processing service corresponding to the service identifier in the logic processing layer according to the feature data to process the event and obtain the processing result information of the target operation event. Finally, the updating module 305 transmits the processing result information to the interface control layer and updates the user interface of the audio / video conferencing control terminal according to the processing result information to display the processing result of the target operation event on the user interface.

[0115] This application enables the control of audio and video conferencing via an audio and video conferencing control terminal comprising an interface control layer and a logic processing layer. Specifically, during the control process, the interface control layer determines the target operation event triggered by the user's operation on the user interface, the characteristic data and service identifier of the target operation event, and transmits the characteristic data and service identifier to the logic processing layer, which then performs the invocation and other processes. This application constructs the audio and video conferencing control terminal in a layered manner, with the logic processing layer and the interface control layer each implementing different functions. Different programming languages ​​are used for development based on the characteristics of the functions implemented by each layer. Compared with the existing technology that uses a single programming language to develop the entire software, this application can significantly improve software development efficiency.

[0116] Furthermore, embodiments of this application also provide a computer device, which can be a terminal or a server, such as... Figure 17 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically: The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 17 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 401 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby providing overall monitoring of the computer device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0117] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0118] The computer device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0119] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0120] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows: The system receives a target operation event triggered by the user interface of the audio / video conferencing control terminal, which includes an interface control layer and a logic processing layer. The interface control layer determines the characteristic data of the target operation event and the service identifier of the event processing service corresponding to the target operation event. It then transmits the characteristic data and service identifier to the logic processing layer. Based on the characteristic data, the logic processing layer calls the event processing service corresponding to the service identifier to process the event and obtain the processing result information of the target operation event. Finally, it transmits the processing result information to the interface control layer, and based on the processing result information, updates the user interface of the audio / video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0121] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0122] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0123] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0124] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the audio / video conferencing implementation methods provided in embodiments of this application. For example, the computer program can execute the following steps: The system receives a target operation event triggered by the user interface of the audio / video conferencing control terminal, which includes an interface control layer and a logic processing layer. The interface control layer determines the characteristic data of the target operation event and the service identifier of the event processing service corresponding to the target operation event. It then transmits the characteristic data and service identifier to the logic processing layer. Based on the characteristic data, the logic processing layer calls the event processing service corresponding to the service identifier to process the event and obtain the processing result information of the target operation event. Finally, it transmits the processing result information to the interface control layer, and based on the processing result information, updates the user interface of the audio / video conferencing control terminal to display the processing result of the target operation event on the user interface.

[0125] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0126] Since the computer program stored in the storage medium can execute the steps in any of the audio and video conferencing implementation methods provided in the embodiments of this application, the beneficial effects that any of the audio and video conferencing implementation methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0127] The audio and video conferencing implementation method, apparatus, storage medium, and computer equipment provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for implementing audio and video conferencing, characterized in that, include: The system receives target operation events triggered by the user interface of the audio / video conferencing control terminal. The audio / video conferencing control terminal includes an interface control layer and a logic processing layer. The logic processing layer and the interface control layer implement different functions, and are developed using different programming languages ​​based on the characteristics of the functions implemented by each layer. The logic processing layer and the interface control layer communicate based on remote procedure call technology. The interface control layer uses the Dart language, and the logic processing layer uses the C++ language. Determining the feature data of the target operation event and the service identifier of the event processing service corresponding to the target operation event in the interface control layer includes: the terminal obtaining the feature data corresponding to the target operation event through a preset event data mapping relationship; and the interface control layer determining the service identifier of the event processing service corresponding to the target operation event from the event service mapping relationship; wherein, the event service mapping relationship includes the service identifiers of the event processing services corresponding to different operation events. The feature data and the service identifier are serialized using the data description language protobuf. The feature data and the service identifier, which are serialized using the data description language protobuf, are transmitted to the logic processing layer. Based on the feature data, the logic processing layer calls the event processing service corresponding to the service identifier to process the event and obtain the processing result information of the target operation event. The processing result information is transmitted to the interface control layer, and the interface control layer updates the user interface of the audio and video conferencing control terminal according to the processing result information, so as to display the processing result of the target operation event on the user interface.

2. The method according to claim 1, characterized in that, The transmission of the feature data and the service identifier, serialized using the protobuf data description language, to the logic processing layer includes: The feature data and the service identifier are serialized in the interface control layer to obtain the event data of the target operation event; The event data of the target operation event is transmitted to the logic processing layer.

3. The method according to claim 2, characterized in that, The transmission of event data of the target operation event to the logic processing layer includes: The event identifier for the target operation event is generated in the interface control layer; Transmit the event data marked by the event identifier to the logic processing layer; The process of transmitting the processing result information to the interface control layer, and updating the user interface of the audio / video conferencing control terminal based on the processing result information to display the processing result of the target operation event on the user interface, includes: Transmit the processing result information marked by the event identifier to the interface control layer; The interface control layer determines the target operation event corresponding to the processing result information based on the event identifier. The interface control layer updates the user interface of the audio and video conferencing control terminal based on the processing result information, so as to display the processing result of the target operation event on the user interface.

4. The method according to claim 3, characterized in that, The step of updating the user interface of the audio / video conferencing control terminal based on the processing result information in the interface control layer, so as to display the processing result of the target operation event in the user interface, includes: Based on the processing result information, the interface control layer constructs the event state information of the target operation event; Based on the event status information, the interface control layer updates the user interface of the audio and video conferencing control terminal to display the processing result of the target operation event on the user interface.

5. The method according to claim 1, characterized in that, The receiving of target operation events triggered by the user interface of the audio / video conferencing control terminal includes: Receive event information of target operation events triggered by the user interface of the audio / video conferencing control terminal; The step of determining the feature data of the target operation event and the service identifier of the event handling service corresponding to the target operation event in the interface control layer includes: The interface control layer extracts feature data of the target operation event from the event information; The service identifier of the event handling service corresponding to the target operation event is determined in the interface control layer.

6. The method according to claim 5, characterized in that, The step of extracting feature data of the target operation event from the event information in the interface control layer includes: The interface control layer adds event information of the target operation event to the controller, and the controller extracts the feature data of the target operation event from the event information. The step of determining the service identifier of the event handling service corresponding to the target operation event in the interface control layer includes: The interface control layer transmits the feature data of the target operation event to the communication interface. The service identifier of the event processing service corresponding to the target operation event is determined through the communication interface according to the first mapping relationship. The first mapping relationship includes the relationship between multiple preset operation events and preset service identifiers. The transmission of the feature data and the service identifier to the logic processing layer includes: The feature data and the service identifier are transmitted to the logic processing layer through the communication interface.

7. The method according to claim 1, characterized in that, The method further includes: When a meeting change message of a preset type is received, the meeting change message is transmitted to the interface control layer; The interface control layer updates the user interface of the audio and video conferencing control terminal according to the meeting change message, so as to display the meeting change information on the user interface.

8. The method according to claim 1, characterized in that, The method further includes: Send audio and video parameter information to the operating system where the audio and video conferencing control terminal is located; The interface control layer receives audio and video stream data transmitted by the operating system based on the audio and video parameter information, and the user interface plays audio and video based on the audio and video stream data.

9. The method according to claim 8, characterized in that, The step of receiving audio and video stream data transmitted by the operating system based on the audio and video parameter information at the interface control layer, and playing audio and video based on the audio and video stream data at the user interface, includes: The operating system generates an audio and video transmission identifier based on the audio and video parameter information, sends the audio and video transmission identifier to the interface control layer, and receives audio and video stream data returned by the server based on a data request message. The data request message is a message sent by the operating system to the server based on the audio and video parameter information. An audio and video transmission channel corresponding to the audio and video transmission identifier is constructed in the interface control layer, and audio and video stream data transmitted by the operating system through the audio and video transmission channel is received. Audio and video playback is performed in the user interface based on the audio and video stream data.

10. The method according to claim 1, characterized in that, The user interface includes a responsive interface and a control interface, and the method further includes: The audio and video conferencing control terminal displays a response interface for the conferencing hardware device. The response interface includes a meeting initiation control corresponding to the conferencing hardware device. The response interface is the interface displayed by the terminal in response to the meeting information of the conferencing hardware device. In response to an operation on the meeting initiation control, the control interface of the target audio-visual conference initiated by the audio-visual conference control terminal is displayed. The control interface includes meeting control controls for the target audio-visual conference and device control controls for the conference hardware device. The meeting control controls are used to change the meeting status of the target audio-visual conference, and the device control controls are used to change the device status of the conference hardware device.

11. The method according to claim 10, characterized in that, The meeting control controls include member audio / video control controls and member management controls, and the method further includes: In response to an operation on the member audio / video control control, a change is triggered in the audio / video output state of the conference equipment used by the conference members in the target audio / video conference; In response to an operation on the member management control, the meeting status of the meeting members in the target audio / video conference is changed.

12. The method according to claim 10, characterized in that, The device control controls include device audio / video control controls and device control exit controls; the method further includes: In response to an operation on the device's audio / video control controls, the audio / video output state of the conference hardware device is triggered to change. In response to an operation on the device control exit control, the audio / video conferencing control terminal releases control of the conferencing hardware device.

13. An audio and video conferencing device, characterized in that, include: A receiving module is used to receive target operation events triggered by the user interface of the audio / video conferencing control terminal. The audio / video conferencing control terminal includes an interface control layer and a logic processing layer. The logic processing layer and the interface control layer implement different functions, and are developed using different programming languages ​​according to the characteristics of the functions implemented by each layer. The logic processing layer and the interface control layer communicate based on remote procedure call technology. The interface control layer uses the Dart language, and the logic processing layer uses the C++ language. The determination module is used to determine the feature data of the target operation event and the service identifier of the event processing service corresponding to the target operation event in the interface control layer, including: the terminal obtains the feature data corresponding to the target operation event through a preset event data mapping relationship, and the interface control layer determines the service identifier of the event processing service corresponding to the target operation event from the event service mapping relationship; wherein, the event service mapping relationship includes the service identifiers of the event processing services corresponding to different operation events; The serialization processing module is used to serialize the feature data and the service identifier based on the data description language protobuf; The transmission module is used to transmit the feature data and the service identifier, which are serialized based on the data description language protobuf, to the logic processing layer. The calling module is used to call the event processing service corresponding to the service identifier in the logic processing layer according to the feature data to perform event processing and obtain the processing result information of the target operation event; An update module is used to transmit the processing result information to the interface control layer, and the interface control layer updates the user interface of the audio and video conferencing control terminal according to the processing result information, so as to display the processing result of the target operation event on the user interface.

14. A storage medium, characterized in that, The storage medium stores multiple computer programs adapted for loading by a processor to perform the steps of the method as claimed in any one of claims 1 to 12.

15. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any one of claims 1 to 12.

Images