Multi-mode fusion intelligent visual intercom communication scheduling method and system

Through the intelligent visual intercom communication scheduling method of multimodal fusion, the problems of single mode and network dependence of existing systems are solved, and dynamic communication scheduling is realized according to the environment and user needs, providing an efficient and flexible communication experience.

CN118984368APending Publication Date: 2024-11-19FUZHOU MILI TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411025146.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Existing visual intercom systems face problems such as insufficient single communication mode, network dependence and lack of intelligent scheduling, resulting in a decline in communication quality when network fluctuations or environmental changes, and cannot meet the diverse needs of users.

Method used

Using multi-modal fusion intelligent visual intercom communication scheduling method, by receiving user requests and obtaining communication environment information, analyzing the adapted communication mode combination, monitoring the communication quality in real time, dynamically adjusting the modal combination to optimize the communication process, Including video calls, voice calls and alarm calls, automatically switch to alternate modes to adapt to network and environment changes.

Benefits of technology

It realizes intelligent judgment based on user needs and environmental conditions, selecting the most suitable communication protocol and modal, providing an efficient and flexible communication experience, ensuring communication quality and stability, and reducing dependence on a single network.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984368A_ABST
    Figure CN118984368A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of communication, in particular to a multi-mode fusion intelligent visual intercom communication scheduling method and a multi-mode fusion intelligent visual intercom communication scheduling system. When a communication request of a user is received, communication environment information of the user is acquired; analyzing an adaptive communication mode combination according to the communication request and the communication environment information of the user; and finally, scheduling corresponding communication modules for intelligent visual intercom communication according to the communication mode combination. According to the method, intelligent judgment can be carried out according to the requirements of communication participants and environmental conditions, the most suitable communication protocol and instruction set are scheduled and selected, such as private local area network communication, standard wide area network communication, analog line communication, operator voice communication and other communication modes, and the video call is intelligently switched. And communication modes such as pure voice communication can be realized, so that more efficient and flexible communication experience can be provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a method and system for multi-modal fusion intelligent visual intercom communication scheduling. Background Art

[0002] As an important part of modern communication technology, the video intercom system integrates cameras and microphones. Its main functions include providing two-way video calls between users and realizing dual recognition of images and sounds, thereby increasing the safety and convenience of the community. The video intercom system has become an important part of the intelligence of many fields such as homes, communities, parks and public security.

[0003] Although video intercom systems have shown strong practicality and convenience in many fields such as homes, communities, parks and public security, they still face challenges such as network dependence, stability of use and environmental adaptability. In addition, with the rise of the concepts of smart homes and smart buildings, the market demand for these systems continues to grow, requiring further innovation and improvement of video intercoms.

[0004] Disadvantage 1: Single communication mode: Many video intercom systems only support single LAN or WAN video and audio communication, which may not be sufficient to meet user needs in some cases; Disadvantage 2: Network dependence: Some modern video intercom systems rely on a stable Internet network connection. When the network is unstable or interrupted, the communication quality will be affected or even unable to connect.

[0005] Disadvantage 3: Lack of intelligent scheduling: Existing systems are usually unable to intelligently select the best communication method (such as mode and protocol, etc.) according to the specific circumstances of the communicating parties. For example, in the case of large network fluctuations, video and voice are automatically switched to ensure communication reliability. Summary of the invention

[0006] In view of this, the object of the present invention is to provide a method and system for multi-modal fusion intelligent video intercom communication scheduling.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: A method for multi-modal fusion intelligent video intercom communication scheduling, comprising the following steps: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

[0008] Furthermore, it also includes: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

[0009] Further, the communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

[0010] Furthermore, it also includes: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

[0011] Furthermore, it also includes: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multimodal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

[0012] A multi-modal fusion intelligent video intercom communication scheduling system includes a processor and a memory, wherein the memory stores a program or instruction, and when the program or instruction is executed by the processor, the following steps are implemented: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

[0013] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

[0014] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: The communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

[0015] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

[0016] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multimodal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

[0017] The beneficial effects of the present invention are: The present invention provides a method and system for multi-modal fusion intelligent video intercom communication scheduling. When receiving a user's communication request, the user's communication environment information is obtained; then, according to the user's communication request and communication environment information, an adaptive communication mode combination is analyzed; finally, the corresponding communication module is scheduled for intelligent video intercom communication according to the communication mode combination. That is, it can make intelligent judgments based on the needs and environmental conditions of the communication participants, and schedule the most suitable communication protocol and instruction set, such as: private LAN network communication (HA80), standard WAN network communication (SIP, RTC), analog line communication and operator voice call (cellular mobile communication, PSTN, VoIP) and other communication modes, intelligently switch video calls, pure voice calls and other communication methods, so as to provide a more efficient and flexible communication experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a flowchart of the steps of a method for multi-modal fusion intelligent video intercom communication scheduling of the present invention; Figure 2 Shown is a structural block diagram of a multi-modal fusion intelligent visual intercom communication scheduling system of the present invention; Description of Figure Numbers: 1-Processor; 2-Memory. DETAILED DESCRIPTION

[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments: like Figure 1 As shown, the present invention provides a method for multi-modal fusion intelligent video intercom communication scheduling, comprising the following steps: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

[0020] From the above description, it can be seen that the present invention has the following beneficial effects: the present invention provides a method for multi-modal fusion intelligent video intercom communication scheduling, which obtains the user's communication environment information when receiving the user's communication request; then analyzes the adaptive communication mode combination according to the user's communication request and communication environment information; finally, the corresponding communication module is scheduled for intelligent video intercom communication according to the communication mode combination. That is, it can make intelligent judgments based on the needs and environmental conditions of the communication participants, and schedule the selection of the most suitable communication protocol and instruction set, such as: private LAN network communication (HA80), standard WAN network communication (SIP, RTC), analog line communication and operator voice call (cellular mobile communication, PSTN, VoIP) and other communication modes, intelligently switch video calls, pure voice calls and other communication methods, so as to provide a more efficient and flexible communication experience.

[0021] Furthermore, it also includes: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

[0022] From the above description, it can be seen that through the above method, dynamic adjustment can be achieved during the communication process to improve user experience.

[0023] Further, the communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

[0024] From the above description, it can be seen that the above specific content is used as the basis for selecting the communication mode combination.

[0025] Furthermore, it also includes: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

[0026] From the above description, it can be seen that through the above method, when performance degradation or failure is detected, it automatically switches to the preset backup mode to ensure that the user experience is not affected.

[0027] Furthermore, it also includes: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multimodal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

[0028] From the above description, it can be seen that the above method can solve the problem of intelligent scheduling during multi-modal simultaneous calls.

[0029] See also Figure 2 The present invention provides a multi-modal fusion intelligent video intercom communication scheduling system, including a processor and a memory, wherein the memory stores a program or instruction, and when the program or instruction is executed by the processor, the following steps are implemented: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

[0030] From the above description, it can be seen that the present invention has the following beneficial effects: the present invention provides a multi-modal fusion intelligent visual intercom communication scheduling system, which obtains the user's communication environment information when receiving the user's communication request; then analyzes the adaptive communication mode combination according to the user's communication request and communication environment information; finally, the corresponding communication module is scheduled for intelligent visual intercom communication according to the communication mode combination. That is, it can make intelligent judgments based on the needs and environmental conditions of the communication participants, and schedule the selection of the most suitable communication protocol and instruction set, such as: private LAN network communication (HA80), standard WAN network communication (SIP, RTC), analog line communication and operator voice call (cellular mobile communication, PSTN, VoIP) and other communication modes, intelligently switch video calls, pure voice calls and other communication methods, so as to provide a more efficient and flexible communication experience.

[0031] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

[0032] From the above description, it can be seen that through the above method, dynamic adjustment can be achieved during the communication process to improve user experience.

[0033] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: The communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

[0034] From the above description, it can be seen that the above specific content is used as the basis for selecting the communication mode combination.

[0035] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

[0036] From the above description, it can be seen that through the above method, when performance degradation or failure is detected, it automatically switches to the preset backup mode to ensure that the user experience is not affected.

[0037] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multimodal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

[0038] From the above description, it can be seen that the above method can solve the problem of intelligent scheduling during multi-modal simultaneous calls.

[0039] Several preferred embodiments or application examples are listed below to help those skilled in the art better understand the technical content of the present invention and the technical contribution made by the present invention relative to the prior art: Preferred embodiment one: The present invention provides a multi-modal fusion intelligent video intercom communication scheduling method, comprising the following steps: S1. Upon receiving a communication request from a user, obtaining the user's communication environment information; the communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; the network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

[0040] S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

[0041] Also includes: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information. In this way, dynamic adjustment can be achieved during the communication process to improve the user experience.

[0042] During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode to ensure that the user experience is not affected.

[0043] Also included is: when a multimodal simultaneous call is detected, entering a simultaneous call multimodal mode; Among them, the simultaneous calling multimodal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

[0044] In this embodiment, the above solution requires the following steps: 1. Receive the user's communication request and obtain the user's communication environment information.

[0045] Through the integrated trigger mechanism and real-time collection of sensor data, a comprehensive assessment of user communication needs is achieved. This process not only covers key factors such as network conditions, ambient light, and noise levels, but also includes detailed information such as user location, device status, and device model. By intelligently analyzing this data, the system can dynamically adjust communication strategies to ensure the best communication experience in all environments.

[0046] The specific implementation method is as follows: 1. Detection of communication trigger mechanism: The system first receives a communication request from the user. Depending on the user's different trigger mechanisms, the system will take different actions [Situation 1: When the user selects the video call interface, the system will call up the default video call module to support high-quality video communication; Situation 2: When the user selects the voice call interface, the system will call up the default voice call module to focus on audio communication; Situation 3: In certain situations, such as alarm triggering, the system will automatically start and give priority to the most stable communication method to ensure timely delivery of information. ] 2. Obtain communication environment information through sensors: The system collects the following communication environment information through built-in or external sensors, including but not limited to the current network status [by detecting network parameters such as bandwidth, latency, packet loss rate and jitter, assessing which type of communication the network environment connection stability is suitable for], ambient light [by measuring the brightness level (Lux), light uniformity, ambient contrast and light source stability of the environment, judging whether it is necessary to adjust the screen brightness or activate other visual aids], noise level [by analyzing the sound level (dB), background noise type, and noise persistence, the system can decide whether to enable the noise reduction function], user location [obtaining the geographic location of users and devices through WAN IP, WIFI, GPS, Beidou, base stations or other positioning technologies in order to provide location-based services or optimize communication paths] and device status [monitoring the operating status of the device, including battery power, signal strength, etc., to ensure device stability during communication], device type number [identifying the device's hardware parameters, microphone, speaker, decoding chip model, decoding protocol version, etc., so that the system can adjust the communication quality and communication method according to the characteristics of the device], etc.

[0047] The environmental condition judgment criteria are as follows in Table 1: Table 1 Based on the user's communication request and environmental information, combined with triggering methods, device capabilities, network conditions and other multi-dimensional information, intelligent analysis is performed to select the most appropriate communication mode combination.

[0048] The specific implementation method is as follows: The system uses advanced algorithms, including but not limited to machine learning, artificial intelligence, and pattern recognition technology, to conduct comprehensive analysis of collected user requests and environmental information. These algorithms are able to identify key influencing factors in the communication process and make intelligent decisions accordingly. Based on the analysis results, the system evaluates and selects the most appropriate combination of communication modes to meet the user's specific needs and environmental conditions, which includes but is not limited to: Network environment adaptability: In the case of a poor network environment, the system will select a more robust communication protocol, such as using analog line communications or automatically switching to a higher quality network connection.

[0049] Under the same environment and noise conditions, when the WAN bandwidth is greater than characteristic value 3, the delay is less than characteristic value 3, the packet loss rate is less than characteristic value 2, and the jitter is less than characteristic value 2, the system will give priority to using the WAN for communication; when the WAN bandwidth is less than characteristic value 1, the delay is greater than characteristic value 1, the packet loss rate is greater than characteristic value 1, and the jitter is greater than characteristic value 1, the system will choose to use the LAN communication instead of the WAN for communication; when the LAN bandwidth is less than characteristic value 1, the delay is greater than characteristic value 1, the packet loss rate is greater than characteristic value 1, and the jitter is greater than characteristic value 1, the system will choose to use the PSTN communication instead of the LAN for communication, and switch the video call to the pure voice call mode. When it is detected that the WAN bandwidth is greater than characteristic value 2, the delay is less than characteristic value 2, the packet loss rate is less than characteristic value 2, and the jitter is less than characteristic value 2, it will switch back to the WAN for communication.

[0050] Noise suppression: In noisy environments, the system will prioritize noise suppression capabilities and use a noise reduction algorithm based on logarithmic minimum mean square error, or LogMMSE, to improve the clarity of voice communications.

[0051] Light compensation: In low-light environments, the system will enhance the video's low-light processing capabilities and improve video quality through image enhancement technology to ensure visibility during video calls.

[0052] When the ambient brightness is lower than characteristic value one, the system will activate the low-light processing capability, increase the brightness of the video communication screen and reduce noise, while lowering the device screen backlight. When the ambient brightness returns to characteristic value two or above, the low-light processing capability will be turned off to restore the normal picture, and the device screen backlight will be increased.

[0053] Geographic location optimization: Based on the user's geographic location, the system will allocate reasonable RTC (real-time communication) server nodes to reduce latency and improve communication efficiency.

[0054] In terms of the economy, safety and accuracy of obtaining device geographic information, the first choice is base station positioning, followed by Beidou positioning (if the device supports this module) or GPS (if the device supports this module).

[0055] Device capability matching: The system will consider the modes and protocol instruction sets supported by the device, and automatically adapt the communication method that best suits the current device to ensure the compatibility and stability of communication.

[0056] AI algorithms optimize communication robustness: The system can use decision tree algorithms to evaluate network conditions and select the best communication protocol; use convolutional neural networks (CNNs) to optimize video quality; and use recurrent neural networks (RNNs) to predict and adapt to users’ communication habits.

[0057] Through this multi-dimensional, intelligent analysis and selection, the system can provide users with a customized communication experience, ensuring the quality and efficiency of communication in any environment. In addition, the system will continue to learn and optimize its algorithms to adapt to changing user needs and communication environments.

[0058] 3. Schedule the corresponding communication module according to the selected mode After determining the optimal communication mode combination, the system will intelligently schedule the corresponding communication modules to achieve the handshake link of voice and video, and dynamically manage the fusion scheduling of multiple communication modes during the communication process. To ensure high availability and flexibility of communication, the backup mode is always on standby, ready to switch seamlessly when the main mode has problems.

[0059] The specific implementation method is as follows: Communication module activation: The system activates the corresponding voice, video or other multimedia communication modules according to the selected communication modality combination. These modules will be configured to work together to support multimodal communication.

[0060] Handshake link establishment: The system will use appropriate protocols and algorithms to establish a handshake link between the two communicating parties. This may include the selection of signaling protocols, encryption negotiation, and negotiation of bandwidth and parameters.

[0061] Multimodal fusion scheduling: The system will monitor the communication quality in real time and adjust the communication mode as needed. For example, when network conditions change, the system may switch from video communication to voice communication, or re-enable video when the network is restored.

[0062] Technical process: The system will follow the following technical process: A. Receiving and parsing communication requests; B. Collecting and analyzing environmental information; C. Intelligent selection and decision-making of communication modes; D. Scheduling of communication modules and establishing links; E. Real-time monitoring and quality control during communication; F. Fault detection and automatic switching of backup modes.

[0063] Judgment criteria: quality, user preference, device capabilities and environmental factors to schedule and switch communication modes.

[0064] Backup mode standby: The system will continuously evaluate the performance of the primary communication mode and automatically switch to the backup mode if any performance degradation or failure is detected, which ensures the continuity and reliability of communication.

[0065] Communication quality feedback: The system will collect feedback information during the communication process, such as user satisfaction, number of communication interruptions, etc., to further optimize the selection of communication modes and scheduling strategies.

[0066] During the communication process, the communication quality is monitored in real time, and the communication mode is dynamically adjusted according to the feedback information. The system not only monitors the communication quality in real time, but also dynamically adjusts the communication mode according to the feedback information to ensure the stability and efficiency of communication. In addition, the system supports multi-modal simultaneous calls and can intelligently select the optimal communication path and mode.

[0067] The specific implementation method is as follows: Real-time monitoring: The system continuously monitors key indicators during the communication process, including but not limited to signal strength, communication delay, packet loss rate, jitter, and user feedback.

[0068] Dynamic adjustment: Based on monitoring results, the system uses preset optimization algorithms and heuristic rules to adjust the communication mode in real time. For example, if network delay is detected, the system may automatically switch from video communication to voice communication to reduce the impact of delay.

[0069] Multi-modal simultaneous calls: The system supports the establishment and management of multiple communication modes at the same time. [Case 1: For example, HA80 and RTC, the system will intelligently select the most appropriate mode for communication based on the current network conditions, user preferences and communication quality. For example, when the network conditions are good, the system may give priority to using RTC for high-definition video communication; when the network is unstable, the system may automatically switch to HA80 to ensure the stability of communication; Case 2: After confirming that the participant has hung up, the system automatically adjusts the communication mode based on the number and needs of the remaining participants. If there are only two participants left, the system will switch from multi-party intercom mode to two-party intercom mode] Standby mode: The system always keeps the standby mode on standby to deal with problems that may occur in the main mode. For example, in the event of network fluctuations or other communication interruptions, the system can quickly activate the standby mode to ensure the continuity of communication.

[0070] Intelligent selection mechanism: The system intelligently selects the optimal communication mode by comprehensively evaluating the performance indicators of each mode and user feedback.

[0071] Feedback loop: The system will collect user feedback and performance data during the communication process to form a feedback loop to continuously optimize the selection and scheduling strategy of the communication mode.

[0072] Communication quality report: The system regularly generates communication quality reports, providing detailed performance indicator analysis and improvement suggestions to help users and system administrators understand the communication status and make corresponding adjustments.

[0073] Through this real-time monitoring and dynamic adjustment method, the system can ensure high-quality communication experience in all environments, while flexibly responding to user needs and environmental changes. In addition, the support of multi-modal simultaneous calls and the intelligent selection mechanism enable the system to make full use of various communication resources to provide users with the best communication services.

[0074] The beneficial effects of this embodiment are: Multimodal integration and intelligent scheduling: Traditional solutions are limited to a single communication mode, while the multimodal fusion method of the present invention integrates multiple communication modes such as private LAN network communication (HA80), standard WAN network communication (SIP, RTC), analog line communication and operator voice calls (cellular mobile communication, PSTN, VoIP), etc., to achieve complementary advantages and disadvantages between modes and provide a comprehensive means of communication. This integration ensures that the system can select the most appropriate communication mode according to the needs and preferences of users in different situations, overcoming the shortcomings of RTC's high dependence on the network under single mode and the small amount of data transmission on analog lines.

[0075] Fault-tolerance mechanism and multi-path communication strategy: The present invention designs a fault-tolerance mechanism and a multi-path communication strategy to reduce the reliance on a single network. In the case of network instability or interruption, the system can automatically switch to a backup network or use a local communication mode, such as PSTN technology, to ensure the continuity and reliability of communication. In addition, the system also uses CDN video service nodes and dynamic load balancing technology, as well as intelligent mode degradation or switching strategies to adapt to changing network conditions.

[0076] Intelligent selection and event priority: The system can comprehensively consider the location of both parties, device status, environmental conditions and user's personalized settings, and intelligently select or switch to the best communication mode. In an emergency, the system can automatically prioritize the use of more reliable communication methods to ensure timely communication of information, and due to the combined capabilities of hardware devices, such as noise suppression algorithms, automatic gain control and image enhancement technology, to adapt to different external environments. Whether in a noisy environment or low light conditions, the system can automatically adjust to enhance voice clarity or improve video quality.

[0077] Preferred embodiment 2: See also Figure 2 The present invention provides a multi-modal fusion intelligent video intercom communication scheduling system, including a processor and a memory, wherein the memory stores a program or instruction, and when the program or instruction is executed by the processor, the following steps are implemented: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

[0078] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

[0079] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: The communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

[0080] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

[0081] Furthermore, when the program or instruction is executed by the processor, the following steps are implemented: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multimodal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

[0082] The present invention has been described by the above-mentioned relevant embodiments and drawings, but the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, modifications and equivalent settings included in the spirit and scope of the claims are all included in the scope of the present invention.

Claims

1. A multi-modal fusion intelligent video intercom communication scheduling method, characterized in that: The following steps are involved: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

2. According to claim 1, a method for multi-modal fusion intelligent video intercom communication scheduling is characterized in that: Also includes: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

3. According to the method of multi-modal fusion intelligent video intercom communication scheduling according to claim 1, it is characterized in that: The communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

4. The method for multi-modal fusion intelligent video intercom communication scheduling according to claim 1 is characterized in that: Also includes: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

5. The method for multi-modal fusion intelligent video intercom communication scheduling according to claim 1 is characterized in that: Also includes: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multi-modal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

6. A multi-modal fusion intelligent video intercom communication scheduling system, characterized in that: The system comprises a processor and a memory, wherein the memory stores a program or an instruction, and when the program or the instruction is executed by the processor, the following steps are implemented: S1. upon receiving a communication request from a user, obtaining the user's communication environment information; S2. Analyze the adaptive communication mode combination according to the user's communication request and communication environment information; S3. According to the communication mode combination, the corresponding communication module is dispatched for intelligent video intercom communication.

7. The multi-modal fusion intelligent video intercom communication scheduling system according to claim 6 is characterized in that: When the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the communication quality is monitored in real time and corresponding feedback information is generated, and the communication mode combination is dynamically adjusted according to the feedback information.

8. The multi-modal fusion intelligent video intercom communication scheduling system according to claim 6 is characterized in that: When the program or instruction is executed by the processor, the following steps are implemented: The communication request includes a video call trigger mechanism, a voice call trigger mechanism and an alarm trigger mechanism; the communication environment information includes network conditions, ambient light, noise level, user location, device status and device model; The network conditions include bandwidth, latency, packet loss rate and jitter; the ambient light includes brightness level, light uniformity, ambient contrast and light source stability; the noise level includes sound level, background noise type and noise persistence; the device status includes battery power, signal strength and positioning data.

9. The multi-modal fusion intelligent video intercom communication scheduling system according to claim 6 is characterized in that: When the program or instruction is executed by the processor, the following steps are implemented: During the intelligent video intercom communication process, the performance of the communication mode combination is continuously detected. When performance degradation or failure is detected, it automatically switches to the preset backup mode.

10. The multi-modal fusion intelligent video intercom communication scheduling system according to claim 6 is characterized in that: When the program or instruction is executed by the processor, the following steps are implemented: When a multi-modal simultaneous call is detected, a simultaneous call multi-modal mode is entered; Among them, the simultaneous calling multi-modal mode is specifically: scheduling the appropriate mode for communication according to the current network conditions, user preferences and communication quality; or automatically adjusting the communication mode according to the number and needs of the remaining participants after confirming that the participant has hung up.

Citation Information

Cited By

  • Dual-mode interphone communication effect evaluation and optimization method based on multi-feature fusion

    CN119450564A