Video dialogue method and system based on Internet of Things communication, and electronic equipment

Abstract

The invention provides a video dialogue method and system based on Internet of Things communication and electronic equipment, and the method comprises the steps: building a communication link between a first terminal and a second terminal, and enabling a cloud platform to obtain network performance data and terminal hardware resource use data after the second terminal responds to a video dialogue request of the first terminal; and predicting a network condition in a preset time based on the network performance data, the terminal hardware resource use data and a pre-trained network quality evaluation model, and adjusting video dialogue quality based on a prediction result. Therefore, the cloud platform predicts the network condition through the pre-trained network quality evaluation model based on the network performance data and the terminal hardware resource use data which are acquired in real time, and adjusts the video conversation quality according to the network condition, so that smooth video conversation experience can be effectively ensured to be provided in different network environments.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things communications, and in particular to a video conversation method, system and electronic equipment based on Internet of Things communications. Background Art

[0002] In traditional building intercom systems, complex wiring, high maintenance costs, and difficulty in system upgrades have always troubled property managers and residents. Especially in high-rise buildings and large communities, traditional wired intercom systems not only have a long construction period, but are also easily affected by environmental factors, resulting in unstable signals and high failure rates. Therefore, with the popularization and development of IoT technology, IoT technology and building intercom systems have begun to merge.

[0003] The existing IoT-based building intercom systems often result in video conversation requests failing to be delivered or freezes or disconnections occurring during the conversation due to differences in network environments and design flaws in the intercom APP itself. This results in poor arrival rate and stability during video conversations, poor video conversation quality, and seriously affects the user experience. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present application provides a video conversation method, system and electronic device based on Internet of Things communication, aiming to solve the technical problem of poor video conversation quality in the prior art. Specifically: In a first aspect, an embodiment of the present application provides a video conversation method based on Internet of Things communication, the method comprising: a first terminal determines first information through a preset request method, generates a video conversation request based on the first information, and sends the video conversation request to a cloud platform; wherein the first information includes first terminal information; the cloud platform verifies the first information in the received video conversation request, establishes a communication link between the first terminal and the second terminal after the verification is passed, and sends the video conversation request to the second terminal after the establishment is successful; the second terminal determines whether to respond to the video conversation request based on the first information in the received video conversation request, and if responsive, determines a data transmission method based on the communication link type, and transmits the video conversation data between the first terminal and the second terminal according to the data transmission method; the cloud platform obtains network performance data and terminal hardware resource usage data, predicts the network status within a preset time based on the network performance data, the terminal hardware resource usage data and a pre-trained network quality assessment model, and adjusts the video conversation quality based on the prediction result.

[0005] In a second aspect, an embodiment of the present application provides a video conversation system based on Internet of Things communication, characterized in that the system includes: a first terminal, a second terminal and a cloud platform; the first terminal is used to determine first information through a preset request method, generate a video conversation request based on the first information, and send the video conversation request to the cloud platform; wherein the first information includes first terminal information; the cloud platform is used to verify the first information in the received video conversation request, establish a communication link between the first terminal and the second terminal after the verification is passed, and send the video conversation request to the second terminal after the establishment is successful; the second terminal is used to determine whether to respond to the video conversation request based on the first information in the received video conversation request, if responded, determine the data transmission mode based on the communication link type, and transmit the video conversation data between the first terminal and the second terminal according to the data transmission mode; the cloud platform is also used to determine whether to respond to the video conversation request based on the first information in the received video conversation request after the second terminal responds to the video conversation request, if responded, determine the data transmission mode based on the communication link type, and transmit the video conversation data between the first terminal and the second terminal according to the data transmission mode.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: one or more processors, a memory, and one or more applications. The one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method of the first aspect.

[0007] In the technical solution provided by the present application, the first terminal determines the first information through a preset request method, generates a video conversation request based on the first information, and sends the video conversation request to the cloud platform; wherein the first information includes the first terminal information; the cloud platform verifies the first information in the received video conversation request, establishes a communication link between the first terminal and the second terminal after the verification is passed, and sends the video conversation request to the second terminal after the establishment is successful; the second terminal determines whether to respond to the video conversation request based on the first information in the received video conversation request, and if it responds, determines the data transmission method based on the communication link type, and transmits the video conversation data between the first terminal and the second terminal according to the data transmission method; the cloud platform obtains network performance data and terminal hardware resource usage data, predicts the network status within a preset time based on the network performance data, the terminal hardware resource usage data and the pre-trained network quality evaluation model, and adjusts the video conversation quality based on the prediction result. Therefore, the cloud platform predicts the network status based on the network performance data and the terminal hardware resource usage data through the pre-trained network quality evaluation model, and adjusts the video conversation quality accordingly, which can effectively ensure that a smooth video call experience can be provided in different network environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0009] Figure 1 A flow chart of a video conversation method based on Internet of Things communication proposed in an embodiment of the present application is shown.

[0010] Figure 2 A structural block diagram of a video conversation system based on Internet of Things communication proposed in an embodiment of the present application is shown.

[0011] Figure 3 A structural block diagram of an electronic device proposed in an embodiment of the present application is shown.

[0012] Figure 4 A structural block diagram of a computer-readable storage medium proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0015] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0016] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0017] In traditional building intercom systems, complex wiring, high maintenance costs, and difficulty in system upgrades have always troubled property managers and residents. Especially in high-rise buildings and large communities, traditional wired intercom systems not only have a long construction period, but are also easily affected by environmental factors, resulting in unstable signals and high failure rates. Therefore, with the popularization and development of IoT technology, IoT technology and building intercom systems have begun to merge.

[0018] The existing IoT-based building intercom systems often result in video conversation requests failing to be delivered or freezes or disconnections occurring during the conversation due to differences in network environments and design flaws in the intercom APP itself, resulting in poor arrival rate and stability during video conversations, which seriously affects the user experience.

[0019] Therefore, in order to solve the problems existing in the prior art, the present application proposes a video conversation method, system and electronic device based on Internet of Things communication, which are described below in conjunction with specific embodiments.

[0020] See also Figure 1 , Figure 1The embodiment of the present application provides a video conversation method based on Internet of Things communication, which is applied to a video conversation system based on Internet of Things communication. It should be noted that the video conversation system mentioned here is part of a building intercom system. Specifically: Step S110: The first terminal determines the first information through a preset request method, generates a video conversation request based on the first information, and sends the video conversation request to the cloud platform.

[0021] The first information includes the first terminal information, which may be, for example, the first terminal information (such as device ID, serial number, location information, etc.), identity identification, etc. of the first terminal. In the embodiment of the present application, the first terminal is a terminal used by the first user (i.e., visitor), and the second terminal is a terminal used by the second user (i.e., resident). It can be understood that the second terminal information is similar to the first terminal information.

[0022] In an embodiment of the present application, the first terminal can be a building terminal device, such as an access control machine, an intercom, etc., or it can be a visitor's own visitor terminal device, such as a mobile phone, a watch, a laptop computer, a tablet computer, etc., and a module for video conversation is integrated in the first terminal.

[0023] In some embodiments, when the first terminal is a building terminal device, the first information can be directly obtained. After the building terminal device is turned on, it first performs a self-check to ensure that each hardware module (such as a communication module, camera, microphone, speaker, etc.) is working properly, and then automatically connects to the nearest base station through the built-in communication module to obtain the network IP address. After that, the building terminal device sends its own terminal information to the cloud platform through the network for registration. The cloud platform records and verifies the device information and assigns it a unique identity and communication authority. Among them, the communication module can be a 4G communication module. It can be understood that the use of wiring-free 4G Internet of Things technology greatly simplifies the deployment process of the building intercom system and reduces the wiring cost and construction difficulty. At the same time, the subsequent maintenance and upgrade of the system are also more convenient and quick.

[0024] In some embodiments, when the first terminal is a visitor terminal device, the visitor terminal device can determine the first information by scanning a device QR code. Specifically, by scanning the QR code, the visitor terminal device obtains authorization to access each hardware module and sends its own terminal information to the cloud platform through the network for registration, and obtains a unique identity and communication authority assigned to it by the cloud platform. The device QR code can be a QR code for a building terminal device.

[0025] It can be understood that since the first terminal and the second terminal integrate a communication module, a camera, a microphone, a speaker, a QR code scanner, etc., remote communication with the cloud platform can be achieved without traditional wiring.

[0026] Furthermore, the first information also includes user information. When the first terminal is a visitor terminal device, the first terminal scans the device QR code to enter the user information filling page; the first information is determined according to the user information filled in by the first user and the first terminal information; and a video conversation request is generated based on the first information.

[0027] The user information includes at least the identity type of the first user and the room number of the second user. The second user is a user using the second terminal. The first terminal and the second terminal conduct a video conversation based on VOIP. Furthermore, the first terminal and the second terminal can conduct a video conversation based on the VOIP WeChat video conversation channel. Visitors can automatically establish a WeChat video connection with the residents of the designated room by scanning the QR code and entering the resident's room number, without the need for manual transfer or complicated operations, which greatly simplifies the communication process between visitors and residents and improves the intelligent level of building management.

[0028] It can be understood that by deeply integrating VOIP technology with the WeChat platform, video conversation requests can be transmitted directly through the WeChat ecosystem, that is, the first terminal can directly call the WeChat of the second terminal without installing an additional intercom APP. This not only improves the convenience and immediacy of user use, but also enables building terminal devices that originally did not have video conversation functions to have video conversation functions.

[0029] In some implementations, the identity type of the first user may be, for example, a delivery man, a courier, a grid worker, a friend, a landlord, etc. Different verification standards may be used in subsequent identity verification based on the identity type of the first user.

[0030] In step S120, the cloud platform verifies the first information in the received video conversation request, establishes a communication link between the first terminal and the second terminal after the verification is successful, and sends the video conversation request to the second terminal after the establishment is successful.

[0031] In an embodiment of the present application, after receiving a video conversation request, the cloud platform needs to first verify the identity of the first user based on the first information, and then try to establish a communication link between the first terminal and the second terminal and send a video conversation request to the second terminal after the verification is passed.

[0032] Specifically, after the first information is verified, the cloud platform first attempts to establish a direct communication link between the first terminal and the second terminal based on the point-to-point penetration technology; if the establishment fails, the cloud platform is used as a transit server to establish an indirect communication link between the first terminal and the second terminal. It should be noted that since the network environment is constantly changing, the communication link established between the first terminal and the second terminal is also dynamically changing.

[0033] In the embodiment of the present application, the point-to-point penetration communication technology is used to reduce the server transfer link, reduce the server load, and improve the communication efficiency; and when the point-to-point connection is unsuccessful, it automatically switches to the cloud platform transfer mode, which can effectively ensure the reliability and stability of communication and realize high-quality video conversation service. In some embodiments, the cloud platform can also verify other key parameters in the video conversation request, such as the requested URL, HTTP method, specific fields in the request header, etc., to ensure that they conform to the expected format and rules, which helps prevent security issues caused by incorrect parameters or malicious construction.

[0034] In other embodiments, the cloud platform can also verify whether the video conversation request complies with the currently supported communication protocol and version requirements. As technology continues to develop, communication protocols and versions may be updated, and the cloud platform needs to ensure that the received request is compatible with them so that subsequent processing can proceed smoothly.

[0035] Step S130, the second terminal determines whether to respond to the video conversation request based on the first information in the received video conversation request, and if so, determines the data transmission mode based on the communication link type, and transmits the video conversation data between the first terminal and the second terminal according to the data transmission mode.

[0036] In the embodiment of the present application, the second terminal responding to the video conversation request means that the resident agrees to have a video conversation with the visitor.

[0037] In the embodiment of the present application, the communication link type includes a direct communication link and an indirect communication link. Specifically, if the communication link is a direct communication link, the video conversation data is transmitted between the first terminal and the second terminal in a direct transmission mode; if the communication link is an indirect communication link, the video conversation data is transmitted between the first terminal and the second terminal in an indirect transmission mode through the cloud platform.

[0038] In a direct communication link, the first terminal and the second terminal directly exchange data packets through the underlying network protocol (such as TCP / IP). Video conversation data (such as video stream, audio stream, control signal, etc.) are directly transmitted with minimal delay and the highest possible bandwidth. The advantage of this transmission method is that it reduces the number of hops and potential delays in data transmission, and may also reduce the risk of data being eavesdropped or tampered with.

[0039] In an indirect communication link, the first terminal first sends data to the cloud platform, which acts as a relay station to receive and temporarily store the data. The cloud platform then forwards the data to the second terminal based on the network address and status information of the second terminal. Although this transmission method increases the complexity and potential delay of data transmission, it has higher flexibility and scalability. The cloud platform can handle multiple tasks such as network routing, load balancing, data encryption and compression to ensure that data can be transmitted reliably and securely in a complex network environment. In addition, the cloud platform can also provide additional services, such as recording video conversations and real-time transcoding to adapt to the playback needs of different devices.

[0040] In some embodiments, the second terminal may have a built-in intelligent response decision module that can automatically evaluate whether to recommend responding to the video conversation request based on the intimacy with the first user, historical conversation records, and the second user's current schedule and emotional state. Furthermore, the module can also provide the second user with customized response rule settings, allowing the second user to adjust the response decision logic according to their own preferences.

[0041] In some embodiments, during the video conversation, the intelligent response decision module can also analyze the second user's expression, tone and other emotional information in real time, and obtain the conversation decision of the second user corresponding to each emotional information, for subsequent optimization of the intelligent response decision module.

[0042] In some implementations, the first terminal and the second terminal may use the TLS protocol to encrypt data transmission during the video conversation to ensure confidentiality and integrity of the data during transmission to prevent illegal attacks and tampering.

[0043] In some implementations, when the resident still does not confirm the visitor's identity after seeing the first information in the video conversation request, a second confirmation can be made with a third user (such as the landlord, property staff, etc.). Specifically, the second terminal generates the second information based on the first information in the video conversation request, and generates a secondary identity verification request based on the second information; wherein the second information includes the first information, the room number of the second user, the room number of the third user, and the second terminal information; the second terminal sends the secondary identity verification request to the third terminal through the cloud platform; the second terminal determines whether to respond to the video conversation request based on the secondary identity verification result sent by the third terminal.

[0044] Step 140, the cloud platform obtains network performance data and terminal hardware resource usage data, predicts the network status within a preset time based on the network performance data, terminal hardware resource usage data and a pre-trained network quality assessment model, and adjusts the video conversation quality based on the prediction result.

[0045] In an embodiment of the present application, the network performance data includes network bandwidth utilization, round-trip delay, packet loss rate and jitter rate during video conversation data transmission; the terminal hardware resource usage data includes the CPU utilization rate, memory occupancy rate and GPU utilization rate of the first terminal and the second terminal.

[0046] Among them, during the video conversation between the first terminal and the second terminal, the cloud platform can obtain network performance data in real time through network sniffing technology, active detection technology, etc.; the terminal hardware resource usage data can be obtained by the first terminal and the second terminal through performance monitoring tools (such as Windows Performance Monitor, Linux top command, special monitoring software), and the first terminal and the second terminal can encapsulate their own terminal hardware resource usage data in the video conversation data for acquisition by the cloud platform.

[0047] In some implementations, when the network performance data and / or the terminal hardware resource usage data are higher than a first preset value, the video resolution and frame rate during the video conversation can be increased; when they are lower than a second preset value, the video resolution and frame rate during the video conversation can be reduced. For example, when the network bandwidth utilization is low, the round-trip delay is low, or the GPU utilization is low, the video resolution and frame rate during the video conversation can be increased.

[0048] In some embodiments, after the second terminal responds to the video conversation request of the first terminal, the cloud platform can determine the first prediction result and the second prediction result based on the network performance data, the terminal hardware resource usage data and the network quality assessment model; adjust the video conversation quality based on the first prediction result, and determine whether to adjust the type of communication link between the first terminal and the second terminal at the next data transmission based on the communication link type and the second prediction result at the time of this data transmission. Among them, the first prediction result is the predicted network change trend within a preset time, and the second prediction result is the confidence level that the predicted network change amplitude at the next transmission moment exceeds the preset amplitude. Further, if the communication link type at the time of this data transmission is a direct communication link, and the confidence level that the network variation amplitude at the next transmission moment exceeds the preset amplitude exceeds the confidence threshold, the communication link type between the first terminal and the second terminal at the next data transmission is adjusted to an indirect communication link.

[0049] It should be noted that the confidence level of predicting that the network deterioration amplitude exceeds the preset amplitude at the next transmission moment can measure the possibility and severity of the sudden deterioration of the network condition. A high confidence level means that preventive measures need to be taken to prevent the video conversation from being interrupted or the conversation quality from being significantly reduced. Therefore, if a direct communication link is currently used, and according to the second prediction result, the confidence level of the network deterioration amplitude exceeding the preset amplitude at the next transmission moment exceeds the preset confidence threshold, the cloud platform can make an intelligent decision to adjust the communication link type for the next data transmission to an indirect communication link, which can effectively reduce the risk of communication interruption caused by a single network link failure.

[0050] In the embodiment of the present application, the network quality assessment model is a fusion model based on a recurrent neural network and an attention mechanism, and the training process of the network quality assessment model includes: obtaining the first training data; performing data cleaning on the initial data for model training, aggregating the cleaned data and extracting target features to obtain the second training data; training the network quality assessment model based on the second training data, and optimizing the model parameters through the back propagation algorithm and the gradient descent method until the training is completed. Among them, the first training data is the historical network performance data and historical terminal hardware resource usage data used for training the model.

[0051] Specifically, data cleaning can remove outliers, duplicate data, and incomplete data records in the first training data to ensure data quality and consistency; aggregation processing includes time aggregation and dimension aggregation. Time aggregation refers to aggregating data at different time points in the first training data after cleaning by time to observe the change trend of the network over time; dimension aggregation can be aggregating data of different dimensions in the first training data after cleaning (such as different network protocols) to reduce the complexity of the data. Target features include statistical features, time series features, and association features; statistical features are used to reflect the overall level and fluctuation of the network based on the statistical values ​​of the first training data after cleaning (such as mean, median, standard deviation, maximum, minimum, etc.); time series features are used to perform time series analysis on the first training data after cleaning, so that the model can determine the law of network changes and predict the future network status; association features are used to enable the model to more comprehensively determine the influencing factors of the network based on the correlation between different data (such as the relationship between network bandwidth utilization and round-trip delay, the relationship between CPU usage and packet loss rate, etc.).

[0052] In some implementations, a weighted learning algorithm of historical data sets can be introduced during model training to strengthen the focus on data points with high impact factors, such as sudden network congestion or hardware resource bottleneck events, so as to improve the model's ability to predict future network quality fluctuations. Furthermore, during model training, the model parameters can be continuously optimized through back propagation algorithms and gradient descent algorithms to minimize prediction errors.

[0053] In some embodiments, if the confidence level of the network deterioration at the next transmission moment exceeds a preset level and exceeds a confidence threshold, a target video image may be obtained and the next video image may be replaced with the target video image. The target video image may be the second video image after the current moment. In this way, the image freeze caused by network deterioration at the next transmission moment may be avoided, and the smoothness of the video conversation image may be ensured.

[0054] In the technical solution provided by the present application, the first terminal determines the first information through a preset request method, generates a video conversation request based on the first information, and sends the video conversation request to the cloud platform; wherein the first information includes the first terminal information; the cloud platform verifies the first information in the received video conversation request, establishes a communication link between the first terminal and the second terminal after the verification is passed, and sends the video conversation request to the second terminal after the establishment is successful; the second terminal determines whether to respond to the video conversation request based on the first information in the received video conversation request, and if it responds, determines the data transmission method based on the communication link type, and transmits the video conversation data between the first terminal and the second terminal according to the data transmission method; the cloud platform obtains network performance data and terminal hardware resource usage data, predicts the network status within a preset time based on the network performance data, the terminal hardware resource usage data and the pre-trained network quality evaluation model, and adjusts the video conversation quality based on the prediction result. Therefore, the cloud platform predicts the network status based on the network performance data and the terminal hardware resource usage data through the pre-trained network quality evaluation model, and adjusts the video conversation quality accordingly, which can effectively ensure that a smooth video call experience can be provided in different network environments.

[0055] See also Figure 2 , Figure 2 A video conversation system 100 based on Internet of Things communication provided by an embodiment of the present application is shown, and the system includes a first terminal 110, a second terminal 120 and a cloud platform 130. Specifically: The first terminal 110 is used to determine the first information through a preset request method, generate a video conversation request based on the first information, and send the video conversation request to the cloud platform 130; wherein the first information includes the first terminal information.

[0056] The cloud platform 130 is used to verify the first information in the received video conversation request, establish a communication link between the first terminal 110 and the second terminal 120 after the verification is successful, and send the video conversation request to the second terminal 120 after the establishment is successful.

[0057] The second terminal 120 is used to determine whether to respond to the video conversation request based on the first information in the received video conversation request, and if so, determine the data transmission mode based on the communication link type, and transmit the video conversation data between the first terminal 110 and the second terminal 120 according to the data transmission mode.

[0058] The cloud platform 130 is also used to determine whether to respond to the video conversation request based on the first information in the received video conversation request after the second terminal 120 responds to the video conversation request. If responded, determine the data transmission method based on the communication link type, and transmit the video conversation data between the first terminal 110 and the second terminal 120 according to the data transmission method.

[0059] In some implementations, the video dialogue system 100 based on IoT communication further includes a third terminal; the third terminal is used to receive the secondary identity verification request sent by the second terminal 120 and send the secondary identity verification result to the second terminal 120.

[0060] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For system-type embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the system embodiment, and will not be repeated one by one in the system embodiment.

[0061] See also Figure 3 , Figure 3 An electronic device 200 proposed in an embodiment of the present application is shown. The electronic device 200 in the present application may include one or more of the following components: a processor 210, a memory 220, and one or more applications, wherein the one or more applications may be stored in the memory 220 and configured to be executed by the one or more processors 210, and the one or more applications are configured to execute the method described in the foregoing method embodiment.

[0062] The processor 210 may include one or more processing cores. The processor 210 uses various interfaces and lines to connect various parts of the entire electronic device 200, and executes various functions and processes data of the electronic device 200 by running or executing instructions, programs, code sets or instruction sets stored in the memory 220, and calling data stored in the memory 220. Optionally, the processor 210 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 210 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 210, but may be implemented separately through a communication chip.

[0063] The memory 220 may include a random access memory (RAM) or a read-only memory (ROM). The memory 220 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 220 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a verification function, a network status prediction function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data created by the electronic device 200 during use, such as first information, second information, network performance data, terminal hardware resource usage data, etc.

[0064] See also Figure 4 , Figure 4 The structure block diagram of a computer-readable storage medium proposed in the embodiment of the present application is shown. The computer-readable storage medium 300 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0065] The computer-readable storage medium 300 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 for executing any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 310 can be compressed, for example, in a suitable form.

[0066] The present application provides a video conversation method, system and electronic device based on Internet of Things communication, wherein the first terminal determines the first information through a preset request method, generates a video conversation request based on the first information, and sends the video conversation request to a cloud platform; wherein the first information includes the first terminal information; the cloud platform verifies the first information in the received video conversation request, establishes a communication link between the first terminal and the second terminal after the verification is passed, and sends the video conversation request to the second terminal after the establishment is successful; the second terminal determines whether to respond to the video conversation request based on the first information in the received video conversation request, and if it responds, determines the data transmission method based on the communication link type, and transmits the video conversation data between the first terminal and the second terminal according to the data transmission method; the cloud platform obtains network performance data and terminal hardware resource usage data, predicts the network status within a preset time based on the network performance data, the terminal hardware resource usage data and a pre-trained network quality evaluation model, and adjusts the video conversation quality based on the prediction result. Therefore, the cloud platform predicts the network status based on the network performance data and the terminal hardware resource usage data through a pre-trained network quality evaluation model, and adjusts the video conversation quality accordingly, which can effectively ensure that a smooth video call experience can be provided in different network environments.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video conversation method based on Internet of Things communication, characterized in that: The method comprises: The first terminal determines the first information through a preset request method, generates a video conversation request based on the first information, and sends the video conversation request to the cloud platform; wherein the first information includes the first terminal information, and the terminal information includes the first terminal information and the second terminal information; The cloud platform verifies the first information in the received video conversation request, establishes a communication link between the first terminal and the second terminal after the verification is successful, and sends the video conversation request to the second terminal after the establishment is successful; The second terminal determines whether to respond to the video conversation request based on the first information in the received video conversation request, and if so, determines a data transmission mode based on a communication link type, and transmits the video conversation data between the first terminal and the second terminal according to the data transmission mode; The cloud platform obtains network performance data and terminal hardware resource usage data, predicts network conditions within a preset time based on the network performance data, the terminal hardware resource usage data and a pre-trained network quality assessment model, and adjusts the video conversation quality based on the prediction result.

2. The video conversation method according to claim 1, characterized in that: The establishing of a communication link between the first terminal and the second terminal includes: Establishing a direct communication link between the first terminal and the second terminal based on a point-to-point penetration technology; If the establishment fails, using the cloud platform as a transit server to establish an indirect communication link between the first terminal and the second terminal; The step of determining a data transmission mode based on the communication link type, and transmitting the video conversation data between the first terminal and the second terminal according to the data transmission mode, includes: If the communication link is a direct communication link, the first terminal and the second terminal transmit the video conversation data in a direct transmission manner; If the communication link is an indirect communication link, the video conversation data is transmitted between the first terminal and the second terminal through the cloud platform using an indirect transmission method.

3. The video conversation method according to claim 2, characterized in that: The first terminal determines the first information in a preset request manner, and generates a video conversation request based on the first information, including: The first terminal enters the user information filling page by scanning the device QR code; Determine first information according to the user information filled in by the first user and the first terminal information; A video conversation request is generated based on the first information; wherein the user information includes at least the identity type of the first user and the room number of the second user, the second user is a user using the second terminal, and the first terminal and the second terminal conduct a video conversation based on VOIP.

4. The video conversation method according to claim 3, characterized in that: The second terminal determines whether to respond to the video conversation request based on the first information in the video conversation request, including: The second terminal generates second information based on the first information in the video conversation request, and generates a secondary identity verification request according to the second information; wherein the second information includes the first information, the room number of the second user, the room number of the third user, and the second terminal information; The second terminal sends the secondary identity verification request to the third terminal through the cloud platform; The second terminal determines whether to respond to the video conversation request based on the secondary identity verification result sent by the third terminal.

5. The video conversation method according to claim 2, characterized in that: The predicting of a network condition within a preset time based on the network performance data, the terminal hardware resource usage data and a pre-trained network quality assessment model, and adjusting the video conversation quality based on the prediction result, includes: Determine a first prediction result and a second prediction result based on the network performance data, the terminal hardware resource usage data and a pre-trained network quality assessment model; wherein the first prediction result is a predicted network change trend within a preset time, and the second prediction result is a confidence level that the predicted network change amplitude at the next transmission moment exceeds a preset amplitude; The video conversation quality is adjusted based on the first prediction result, and based on the communication link type during the current data transmission and the second prediction result, it is determined whether to adjust the communication link type between the first terminal and the second terminal at the next transmission moment.

6. The video conversation method according to claim 5, characterized in that: The determining, based on the communication link type during the current data transmission and the second prediction result, whether to adjust the communication link type between the first terminal and the second terminal at the next transmission moment includes: If the communication link type during this data transmission is the direct communication link, and the confidence level that the network deterioration amplitude at the next transmission moment exceeds the preset amplitude exceeds the confidence threshold, the communication link type between the first terminal and the second terminal at the next transmission moment is adjusted to the indirect communication link.

7. The video conversation method according to any one of claims 5 to 6, characterized in that: The network quality assessment model is a fusion model based on a recurrent neural network and an attention mechanism. The training process of the network quality assessment model includes: Acquire first training data; wherein the first training data is historical network performance data and historical terminal hardware resource usage data used for training the model; Performing data cleaning on the initial data for model training, performing aggregation processing on the cleaned data and extracting target features to obtain second training data; wherein the aggregation processing includes time aggregation and dimension aggregation, and the target features include statistical features, time series features and correlation features; The network quality assessment model is trained based on the second training data, and the model parameters are optimized by a back propagation algorithm and a gradient descent method until the training is completed.

8. A video dialogue system based on Internet of Things communication, characterized in that: The system comprises: a first terminal, a second terminal and a cloud platform; The first terminal is used to determine the first information through a preset request method, generate a video conversation request based on the first information, and send the video conversation request to the cloud platform; wherein the first information includes the first terminal information; The cloud platform is used to verify the first information in the received video conversation request, establish a communication link between the first terminal and the second terminal after the verification is successful, and send the video conversation request to the second terminal after the establishment is successful; The second terminal is used to determine whether to respond to the video conversation request based on the first information in the received video conversation request, and if so, determine a data transmission mode based on the communication link type, and transmit the video conversation data between the first terminal and the second terminal according to the data transmission mode; The cloud platform is further used to determine whether to respond to the video conversation request based on the first information in the received video conversation request after the second terminal responds to the video conversation request; if so, determine the data transmission method based on the communication link type, and transmit the video conversation data between the first terminal and the second terminal according to the data transmission method.

9. The video dialogue system according to claim 8, characterized in that: The system also includes a third terminal; the third terminal is used to receive the secondary identity verification request sent by the second terminal, and send the secondary identity verification result to the second terminal.

10. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1-7.

Images