A cell building visual intercom system using internet of things technology

Abstract

The application relates to the technical field of building visual intercom, and discloses a community building visual intercom system using Internet of Things technology, which comprises an image acquisition and preprocessing module, a semantic analysis and quantization module, a channel state prediction module and a bimodal transmission control module. The system firstly pre-processes the collected images, calculates the semantic importance factor of the region of interest by using semantic segmentation, constructs a channel transmission resistance model based on physical parameters, introduces a trend sensitivity coefficient to predict the channel transmission resistance at the next moment, and executes bimodal decision according to the prediction result: when the network fluctuates, the quantization parameters are dynamically adjusted based on the semantic weight to preferentially guarantee the face clarity; when the network is extremely congested, the background freezing strategy is switched to, and only the dynamic ROI is transmitted to reconstruct the picture in combination with the local high-definition background reference frame. The application solves the problems of video lag and interruption in a weak network environment, and realizes the optimal balance between visual experience and network status.

Description

Technical Field

[0001] This invention relates to the field of building visualization intercom technology, specifically a community building visualization intercom system utilizing Internet of Things (IoT) technology. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and mobile internet technologies, smart community construction has become an important part of urban modernization. As a crucial component of community security and convenient services, the building-level visual intercom system, relying on network communication technology, enables real-time two-way voice and video interaction between visitors and residents, enhancing safety and convenience. In this system, the real-time transmission quality of video data directly affects the user experience, especially in the core function of verifying visitor identity; clear and smooth video streams are key to the system's stable operation.

[0003] In existing technologies, building visualization intercom systems typically employ common video coding standards such as H.264 or H.265, and transmit audio and video data streams over IP networks via RTP / RTCP protocols. To cope with network bandwidth fluctuations, existing solutions often use feedback-based adaptive rate control (ABR) technology. This means the system dynamically adjusts the output bitrate or resolution of the video encoder based on the packet loss rate or round-trip time (RTT) reported by the receiver. When poor network conditions are detected, the system attempts to reduce video quality to decrease data volume, striving to maintain connection stability.

[0004] However, the existing technologies still have some shortcomings in practical applications, especially in scenarios with complex and ever-changing network environments: When faced with network fluctuations, existing building intercom systems typically only begin to reduce the bitrate after the video has already stuttered or become blurry. However, by this time, the channel is often already congested, and this passive response mechanism exhibits significant lag. Especially in weak network environments such as elevators and underground parking garages, simple global compression strategies often lead to complete video interruption when the encoding limit is reached. Therefore, there is an urgent need for a community building visualization intercom system utilizing IoT technology to solve these problems. Summary of the Invention

[0005] To address the problems in related technologies, this invention provides a community building visual intercom system utilizing Internet of Things (IoT) technology to overcome the aforementioned technical problems in existing related technologies.

[0006] To solve the aforementioned technical problem, the present invention is achieved through the following technical solution: In a first aspect, embodiments of the present invention provide a community building visualization intercom system utilizing Internet of Things (IoT) technology, specifically comprising: an image acquisition and preprocessing module, used to construct a multi-dimensional data acquisition mechanism with temporal memory capability, capture video streams, and perform histogram equalization preprocessing on the images; a communication module, used to detect network parameters via the RTCP protocol, and extract a frame without dynamic targets in the image as a high-definition background reference frame; a semantic analysis and quantization module, used to perform semantic segmentation on video frames, identify regions of interest and background regions, and calculate semantic importance factors based on the relative information density of regions of interest, quantifying the value distribution of the image content; a channel state prediction module, used to construct a channel transmission resistance model based on network parameters, calculate the current channel transmission resistance coefficient, and predict the channel transmission resistance coefficient at the next moment based on the changing trend of the resistance coefficient; and a dual-mode transmission control module, used to set a high-risk resistance threshold, compare the channel transmission resistance coefficient at the next moment with the high-risk resistance threshold; if the predicted coefficient is less than or equal to the threshold, execute an adaptive weight allocation strategy; if the predicted coefficient is greater than the threshold, execute a background freezing strategy.

[0007] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the histogram equalization preprocessing step includes: converting the original image from the RGB color space to the YCrCb color space and extracting the luminance component. And divided into Divide the data into non-overlapping rectangular sub-blocks; calculate the grayscale histogram of each sub-block and set a clipping threshold. For gray levels with excessive frequency, the number of clipped pixels is evenly distributed across all gray levels in the histogram of that sub-block to obtain a smoothed histogram. The cumulative distribution function of each sub-block is calculated to establish a local gray-level mapping table. Bilinear interpolation is used to remap the pixels to obtain the corrected brightness value. Finally, the corrected brightness value is merged with the original chroma component and sent to the encoder in YUV format.

[0008] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the formula for calculating the semantic importance factor is as follows: ; In the formula, As a semantic importance factor, This is a dynamic safety net factor. The area percentage of the region of interest. This represents the confidence level based on the output of a lightweight convolutional neural network.

[0009] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the formula for calculating the channel transmission resistance coefficient is as follows: ; In the formula, For the current moment The channel transmission resistance coefficient, For a moment Network jitter value, For a moment Round-trip delay, To the maximum tolerable delay, To dilute the weight, For delay weighting, This is the sensitivity adjustment index.

[0010] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the formula for calculating the channel transmission resistance coefficient at the next moment is: ; In the formula, This represents the channel transmission resistance coefficient at the next time step. For the current moment The channel transmission resistance coefficient, For time intervals, The rate of change of the channel transmission resistance coefficient. This represents the trend sensitivity coefficient. The trend sensitivity coefficient The calculation formula is: ;in, For video capture frame rate, Configure the buffer capacity for the receiving terminal.

[0011] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the method for determining the high-risk resistance threshold is as follows: The physical critical state of network transmission is defined as the sum of weighted network jitter and round-trip delay equaling the maximum tolerable delay constant, at which point the normalized network pressure value is 1; the high-risk resistance threshold is then calculated. ,in This is the high-risk resistance threshold.

[0012] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the specific steps of the adaptive weight allocation strategy include: based on semantic importance factors... and channel transmission resistance coefficient Calculate the dynamic bitrate weight of the region of interest ; Allocate ROI target bitrate based on dynamic bitrate weights and background target bitrate Based on the gradient energy of the ROI region and background gradient energy Calculate and adjust the quantization parameters of the ROI region respectively. and background area quantization parameters The calculation of the quantization parameters introduces a boundary truncation function to limit the values ​​to the range of the standard QP.

[0013] As a preferred embodiment of the community building visualization intercom system utilizing Internet of Things technology described in this invention, the dynamic bitrate weight of the region of interest... The calculation formula is: ; in, The background suppression index is used.

[0014] As a preferred embodiment of the community building visualization intercom system utilizing IoT technology described in this invention, the specific steps of the background freezing strategy include: at the transmitting end, extracting the region of interest using semantic masking and performing low-bitrate encoding to generate an instruction packet containing reconstruction instructions and region of interest data, which is then sent to the receiving end; at the receiving end, parsing the instruction packet, retrieving a pre-stored high-definition background reference frame from the local cache, and real-time superimposing and fusing the received dynamic region of interest onto the high-definition background reference frame through coordinate mapping, and performing feathering smoothing processing on the edges to render the composite image.

[0015] Secondly, embodiments of the present invention provide a building visual intercom terminal, specifically including: a wide-angle camera module, a high-definition display screen, an Internet of Things (IoT) communication module, a main controller, and a community building visual intercom system utilizing IoT technology; the main controller is connected to the wide-angle camera module, the high-definition display screen, and the IoT communication module respectively, runs the community building visual intercom system utilizing IoT technology, and controls the wide-angle camera module to collect visitor video streams; the IoT communication module is used to monitor network channel status and transmit encoded data packets, and the high-definition display screen is used to display reconstructed or decoded video images on the resident's end.

[0016] The present invention has the following beneficial effects: 1. This invention, by constructing a channel transmission resistance model and trend prediction mechanism, differs from the passive response mode of existing technologies that rely on packet loss rate or round-trip delay feedback. It can predict the transmission risk of the next moment in advance based on the changing trend of the channel resistance coefficient before the network actually experiences severe congestion. This allows the system to make proactive adjustments before congestion occurs, avoiding screen stuttering caused by response lag and ensuring a smooth transition of the video stream.

[0017] 2. This invention introduces a dual-modal strategy of background freezing and adaptive weight allocation. When the network is predicted to be in a state of extreme congestion, the system no longer forces the transmission of the full frame. Instead, it uses semantic analysis technology to extract and transmit only the region of interest with a very low amount of data (such as the visitor's face). At the receiving end, it merges the region with a pre-cached high-definition static background, thereby ensuring that the core function of visitor identification can still be used normally under extremely low bandwidth conditions and preventing the communication connection from being disconnected.

[0018] 3. This invention utilizes a lightweight convolutional neural network to perform semantic segmentation on video frames, distinguishing between regions of interest (ROI) and background regions, and dynamically allocating bitrate weights based on the difference in information density between the two. When network fluctuations limit total bandwidth, the system automatically sacrifices the image quality of background regions, concentrating bitrate resources on encoding key information such as faces. This ensures that even with degraded network quality, visitor facial features remain clear and sharp, fulfilling the core function of building intercom systems in verifying visitor identity.

[0019] 4. This invention introduces adaptive histogram equalization processing based on luminance components before video encoding. By independently processing the luminance components in the YCrCb space and using a local histogram equalization algorithm that limits contrast, it solves the problems of dark faces under backlight and excessive noise at night, improves the image quality seen by users on indoor terminals, and increases the recognition accuracy under extreme lighting conditions.

[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the modules of the community building visualization intercom system utilizing Internet of Things (IoT) technology provided by the present invention.

[0023] Figure 2 The flowchart of the community building visual intercom method using Internet of Things technology provided by the present invention.

[0024] Figure 3 This is a schematic diagram of the building video intercom terminal architecture provided by the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1 Existing building intercom systems often exhibit significant lag when facing network fluctuations; typically, the system only begins to reduce the bitrate after the video has already stuttered or become blurry, by which time the channel is often already congested. Especially in weak network environments such as elevators and underground parking garages, traditional compression strategies can easily lead to complete video interruption when they reach their encoding limits.

[0027] To solve the above technical problems, such as Figure 1As shown, Embodiment 1 of the present invention provides a community building visualization intercom system utilizing Internet of Things (IoT) technology, specifically including: an image acquisition and preprocessing module, a semantic analysis and quantization module, a channel state prediction module, and a dual-modal transmission control module. The image acquisition and preprocessing module is used to construct a multi-dimensional data acquisition mechanism with temporal memory capabilities. The camera module acquires raw image sequences at a preset frame rate and improves image robustness under backlight or low-light conditions through histogram equalization preprocessing. Simultaneously, the communication module detects round-trip delay and network jitter values ​​through the RTCP protocol and synchronizes the video frame timestamps with the network parameter timestamps to generate a temporal feature vector containing the video stream and channel state. The semantic analysis and quantization module uses a lightweight convolutional neural network deployed on the edge to perform semantic segmentation on video frames, quickly locating regions of interest (ROIs) containing faces, shoulders, and hands, and marking the remaining areas as background areas. By calculating the gradient energy ratio of the ROI region to the entire image, combined with area proportion and confidence, the semantic importance factor is quantified to reflect the value distribution of the image content, providing data support for subsequent differentiated bitrate allocation. The channel state prediction module is used to construct a channel transmission resistance model based on physical parameters, introducing the maximum tolerable delay constant as a benchmark, and fusing network jitter and round-trip delay into a standardized network pressure value. It calculates the channel transmission resistance coefficient at the current moment using an exponential decay function, and predicts the channel transmission resistance coefficient at the next moment based on the rate of change of the resistance coefficient, thereby detecting transmission risks in advance before severe network congestion actually occurs. The dual-mode transmission control module is used to perform dual-mode decision-making based on the predicted channel transmission resistance coefficient. When the predicted coefficient is less than the high-risk resistance threshold, an adaptive weight allocation strategy is executed, dynamically adjusting the quantization parameters of the ROI region and background region according to the semantic importance factor, concentrating bitstream resources on key information. When the predicted coefficient is greater than the high-risk resistance threshold, a background freezing strategy is executed, transmitting only the dynamic ROI region with extremely low bitrate, and using pre-stored high-definition background reference frames for image reconstruction and fusion at the receiving end.

[0028] Specific embodiment 1 is as follows: In a large-scale modern smart community, a visual intercom host is deployed in the entrance lobby and underground garage entrance of the high-rise residential buildings. An embedded wide-angle camera module is used to collect the original video stream (preset frame rate 25fps). The communication module is configured to monitor the network round-trip time (RTT) and jitter value at high frequency through the RTCP protocol, and connect to the smart display terminal in the resident's room to provide the data foundation for embodiment 1 of the present invention.

[0029] In the specific implementation of the above-mentioned Embodiment 1, firstly, a multi-dimensional data acquisition mechanism with temporal memory capability is constructed, and histogram equalization preprocessing is performed on the video stream. This method automatically identifies and optimizes image details in backlit or low-light environments by independently processing the luminance component in the YCrCb space, solving the common problems of backlit faces appearing black and excessive noise at night at building entrances. Secondly, semantic segmentation is performed on the video frames to identify regions of interest (ROIs) and calculate semantic importance factors. This method uses a lightweight convolutional neural network to quickly locate key regions such as faces and shoulders, and introduces relative information density gain to correct the bias of judging importance solely based on physical area. This ensures that even in scenarios where visitors are far away, the system can still lock them as core transmission objects based on their high-frequency texture features, avoiding the loss of key information caused by traditional global compression strategies. Then, a channel transmission resistance model is constructed based on physical parameters, and the channel transmission resistance coefficient at the next moment is predicted using the rate of change. This method uses exponential decay to simulate the nonlinear decline in user experience with network fluctuations, and uses a trend amplification mechanism to detect risks in advance before severe network congestion actually occurs, overcoming the passive lag of existing technologies that rely on packet loss rate feedback, and providing a quantitative and timely decision-making basis for proactive system adjustment. Finally, bimodal decision-making is performed based on the predicted resistance coefficient, implementing either an adaptive weight allocation strategy or a background freezing strategy. This method proactively sacrifices background image quality to concentrate bitrate to ensure clear faces during network fluctuations, and maintains connection by transmitting dynamic ROI and reusing local static backgrounds during extreme congestion, achieving an optimal balance between visual experience and network status, and solving the problem of video stuttering or complete communication interruption in weak network environments.

[0030] Furthermore, to better illustrate the technical solution of Embodiment 1 of the present invention, such as... Figure 2 As shown, this paper describes a community building visual intercom system in detail, using a method that utilizes Internet of Things (IoT) technology. The specific details include the following: S1. Initial stage: Construct a multi-dimensional data acquisition mechanism with temporal memory capability, and simultaneously capture video streams and channel states. This includes the following sub-steps: S11. The camera module acquires raw image sequences at a preset frame rate (e.g., 25fps). However, to avoid blocking the main thread due to subsequent high-load calculations, the original image is transferred to the circular buffer of the image processing unit via DMA (Direct Memory Access). During this process, considering that building intercom systems are usually in backlit or low-light environments, the system automatically performs histogram equalization preprocessing on the image to improve the robustness of subsequent recognition algorithms under extreme lighting conditions.

[0031] For example, to address the common issues of darkened faces in backlit or low-light environments at building entrances and excessive noise at night, histogram equalization preprocessing is performed on the image. The specific implementation steps are as follows: S1-1 Image Preprocessing Sub-step: Adaptive Equalization Processing Based on Luminance Component To address the common problems of backlit faces appearing dark and excessive noise at building entrances, the system performs the following hierarchical processing before storing the original image in the buffer: S11-1, the image processing unit first reads the original RGB format video frames. To avoid color distortion (such as facial color distortion) caused by directly processing RGB, the system converts the image from the RGB color space to the YCrCb color space through a linear transformation. The luminance component is then extracted. As data to be processed (containing grayscale information), preserve the chroma components. It remains unchanged and is used for subsequent color restoration.

[0032] S11-2. For local differences in brightness between the background and the face in backlit scenes, the system does not calculate the histogram of the entire image, but instead calculates the brightness component. Divided into Non-overlapping rectangular sub-blocks (e.g.) (Grid). The system computes the grayscale histogram of each sub-block in parallel. ,in Represents grayscale levels (0-255).

[0033] S11-3. To prevent the sensor noise from being amplified due to excessive contrast stretching in low-light areas (such as dark areas at night), the system presets a clipping threshold. Iterate through the histogram of each sub-block, for those with a frequency exceeding... The gray levels are calculated, and the portion exceeding the cutoff values ​​is cropped. The total number of cropped pixels is counted and evenly distributed across all gray levels of the histogram of that sub-block, thus reconstructing a smoothed histogram. This is used to avoid image whitening or noise bursts caused by excessive concentration of a single gray level.

[0034] S11-4. Based on the smoothed histogram, calculate the cumulative distribution function (CDF) of each sub-block and establish a local grayscale mapping table. To eliminate the obvious block effect between sub-blocks, bilinear interpolation is used instead of directly using the mapping function of the sub-block during pixel remapping. For any pixel in the image... The system performs a weighted calculation based on the mapping function of the centers of its four adjacent sub-blocks to obtain the final corrected brightness value. .

[0035] S11-5, Enhanced luminance component Compared with the original chromaticity components The images are then merged and converted back to the RGB color space, or directly fed into the encoder in YUV format. This results in an output image that significantly improves the contrast of dark areas (visitor's face) while preserving background details.

[0036] S12. The communication module runs silently in the background, while simultaneously probing network parameters at a high frequency (e.g., every 50ms) via RTCP (Real-Time Transmission Control Protocol). To capture the dynamic characteristics of network changes, the system not only records the current moment... It can reflect the round-trip delay due to physical distance and routing congestion. And network jitter values ​​that reflect signal stability And will also recently Historical data at each moment (e.g., within the past 2 seconds) is stored in the trend buffer.

[0037] S13. Perform synchronization alignment operation, and set the timestamps of the video frames. Timestamps of network parameters Perform matching, search The smallest data pairs form the time series feature vector. During this process, the system will pre-check the stability of the video stream; if continuous [frames / images] are detected... If there are no moving targets in the frame, i.e. no one is passing by, then one frame is automatically extracted as the high-definition background reference frame. A unique hash identifier is generated and pre-sent to the receiving end for caching using idle bandwidth.

[0038] S2. Perform semantic segmentation on the video frames, identify regions of interest (ROIs), and calculate semantic importance factors. Quantifying the value distribution of image content includes the following sub-steps: S21. To achieve intelligent analysis on embedded access control terminals with limited computing power, a lightweight convolutional neural network is deployed on the edge, and video frames are processed... Input a pre-built lightweight convolutional neural network to perform inference on each frame of the image, quickly locate the rectangular bounding boxes containing the face, shoulders and hands, and mark them as ROI regions, while the remaining regions are marked as background regions (BG). S22. Perform semantic region segmentation and dynamically calculate the minimum guarantee coefficient based on relative information density. This generates the final semantic importance factor. The specific implementation steps are as follows: S22-1. Identifying video frames using a lightweight convolutional neural network. The ROI region in the diagram is used to count the number of pixels within that ROI region. and the total number of pixels in the image Calculate the area ratio Simultaneously, obtain the confidence level of the output of the lightweight convolutional neural network. .

[0039] S22-2. In order to objectively quantify the texture complexity of the image, the system will analyze the video frames. The image is converted to grayscale and then convolved with the entire image using the Sobel operator to generate a gradient map. The value of each pixel in this gradient map It represents the sharpness of the edge of that point, that is, the amount of information.

[0040] S22-3. Integrate (accumulate) the gradient values ​​within the ROI region and the entire map region respectively to calculate the ROI gradient energy. and the gradient energy of the whole image Then calculate the information proportion. This ratio objectively reflects the true weight of the ROI region in terms of visual information content.

[0041] S22-4. Calculate the difference between the information ratio and the area ratio to obtain the relative information density gain, i.e., the dynamic minimum guarantee coefficient. .

[0042] S22-5, Using the calculated dynamic safety net coefficient The final composite index is: ,and The value range is [0,1].

[0043] Specifically, for example: suppose a visitor stands in front of a pure white wall, and their face only occupies 10% of the image (i.e., However, since white walls have no texture (gradient is almost 0), faces may contain 90% of the texture details in the entire image. ).at this time Assuming Substitute into the formula to calculate This allows the system to correctly determine that "although the face is small, it is the only meaningful content in the image," thus assigning it a very high bitrate weight.

[0044] In this embodiment 1, a semantic importance factor is introduced. This approach deeply quantizes the value density of image content, abandoning the passive strategy of global uniform compression that relies solely on resolution or bitrate in existing technologies; in particular, it utilizes relative information density gain. The compensation mechanism corrects the bias of judging importance solely by physical area, ensuring that even in scenarios where the visitor is far away (with a small physical proportion of the face), the system can still identify the visitor as a core transmission object based on its high-frequency texture features. This step provides a mathematical basis for subsequent differentiated bitrate allocation, thereby solving the security risk of being unable to confirm the visitor's identity due to overall image blurring in weak network environments.

[0045] S3. Construct a channel transmission resistance model based on network parameters and calculate the channel transmission resistance coefficient. Objectively assess the ease of passing through the network environment, including the following sub-steps: S31. Considering the different destructive effects of latency (milliseconds) and jitter (milliseconds but with large fluctuations) on the video experience, a maximum tolerable latency constant for the system is introduced. As the base denominator, and through a weighted summation, and Integrate into standardized network stress values The specific implementation steps are as follows: S31-1. During the system startup initialization phase, the system reads the buffer capacity configuration of the receiving terminal. (Frame count) and factory-specified end-to-end hardware processing time (ms). Simultaneously, it invokes pre-stored ITU-T interactive experience standard constants. .

[0046] S31-2 Calculate the maximum tolerable delay This indicates the network latency budget limit for the current device in this session. For high-performance terminals, the system will automatically increase this limit. It allows the connection to be maintained under poor network conditions; however, for low-performance terminals, the system will tighten the connection. This allows optimization strategies to be triggered earlier.

[0047] S31-3, calculated Using the base denominator, a weighted summation method is used to calculate the currently captured values. and Fusion. Set jitter weights. and delay weight (satisfy ), calculate network stress value : ; This step maps two network metrics with inconsistent units and different fluctuation characteristics into a standardized stress index linked to hardware capabilities.

[0048] For example, jitter weight and delay weight The determination method is as follows: First, the system reads the buffer depth configuration and current encoding frame rate of the receiver hardware, and calculates the maximum jitter tolerance limit for physical buffer overflow. And retrieve the maximum latency tolerance limit specified in the international interaction standard. Based on the principle that the lower the tolerance, the higher the sensitivity, the following calculations are performed respectively. and The reciprocal of the product is used as the jitter sensitivity factor and the time delay sensitivity factor; the two sensitivity factors are normalized, and their proportion in the total sensitivity is calculated to generate the jitter weight. With delay weight This ensures that the system can automatically and dynamically adjust its focus on different network metrics based on the physical limitations of the hardware itself.

[0049] S32. To simulate the nonlinear characteristic of user experience rapidly declining as network quality deteriorates, an exponential decay channel transmission resistance model is constructed, and the channel transmission resistance coefficient is calculated. The formula is as follows: ; ; in, The sensitivity adjustment index, The range of values ​​is .

[0050] Specifically, for example: setting .enter Substitute into the formula to calculate the channel transmission resistance coefficient. .at this time This indicates that the network resistance is low and the channel is unobstructed.

[0051] S33. Calculate the rate of change of the channel transmission resistance coefficient using the differential method: ; In the formula, For the past The channel transmission resistance coefficient recorded at all times; For the current moment With the past moment The time interval.

[0052] S34. Based on the changing trend of channel transmission resistance coefficient Predict the channel transmission resistance coefficient at the next moment. : ; In the formula, The range of values ​​is ; This is the trend sensitivity coefficient (seconds). The formula amplifies the trend effect through an exponential term: when the network deteriorates sharply (i.e., when...)... And when the value is large, The predicted value will spike rapidly, forcing the system to anticipate the risk of congestion; when the network stabilizes, the predicted value will revert to the current value.

[0053] Furthermore, in order to adapt to hardware terminals with different performance levels, the trend sensitivity coefficient... Dynamically calculated based on the physical characteristics of the current session's buffer: ; in, The video capture frame rate is set in step S11.

[0054] Specifically, for example: reading the system initialization configuration, camera capture frame rate. Receiver hardware buffer capacity (Corresponding to a physical buffer duration of 600ms), calculate the trend sensitivity coefficient: Assuming sampling interval The system records the previous moment ( Channel transmission resistance coefficient At the current moment ( coefficient Calculate the rate of change of the drag coefficient: Substitute the values ​​into the formula to calculate the channel transmission resistance coefficient at the next time step. .

[0055] In this embodiment 1, the physical characteristic of user experience nonlinearly decreasing with network fluctuations is simulated by exponential decay, and a channel transmission resistance model is constructed. Discrete and heterogeneous network jitter and round-trip delay parameters are mapped to a channel transmission resistance coefficient that is linked to hardware tolerance. And combined with the exponential amplification mechanism of the rate of change, the transmission state at the next moment is predicted, that is... This network congestion early warning mechanism ensures that the system can detect risks before channel congestion actually occurs, overcoming the lag of traditional related technologies based on packet loss rate feedback, and providing a quantitative basis for subsequent dual-mode strategy switching.

[0056] S4. Predicted channel transmission resistance coefficient Performing bimodal decision-making specifically includes the following steps: S41. Set high-risk resistance thresholds. The system compares and predicts the channel transmission resistance coefficient in real time. High-risk resistance threshold Size relationship: like If the network is determined to be within a controllable fluctuation range, an adaptive weight allocation strategy is executed. like If the network is determined to be in a state of extreme congestion, the system immediately executes a background freeze policy.

[0057] For example, in this embodiment 1, the high-risk resistance threshold The determination method is as follows: Define the physical critical state of network transmission, that is, when the sum of the weighted network jitter and round-trip delay accurately exhausts the system's maximum tolerable delay constant. When the system delay budget reaches its saturation point, the normalized network pressure value is constant at 1. This critical pressure value of 1 is then used as input and substituted into the channel transmission resistance coefficient calculation formula. Any power of is Based on the mathematical properties, the high-risk resistance threshold was calculated. .like ,because If the network is determined to be within a controllable fluctuation range, an adaptive weight allocation strategy is executed.

[0058] S5. Execute the adaptive weight allocation strategy, which includes the following sub-steps: S51. Calculate the dynamic bitrate weight of the ROI region. : ; In the formula, The background suppression index is used to control the degree of nonlinear response of the code rate weight as channel resistance changes.

[0059] Furthermore, to ensure that the adjustment range of the algorithm strictly matches the physical capability boundary of the video encoder, the background suppression index... Based on the objective calculation of the encoder's quantization parameter range: ; in, and These are the maximum and minimum quantization parameters configured for the current video encoder (e.g., 51 and 0 under the H.264 standard). This is a physical constant (approximately 0.1155) derived from coding rate distortion theory. This determination method ensures that when the channel transmission resistance coefficient... When approaching the high-risk threshold (i.e., the network is about to fail), the algorithm's suppression instructions for the background region just reach the compression limit that the encoder can execute. achieve This achieves a perfect closed loop of software and hardware capabilities, avoiding [the negative consequences of] [the lack of] [the following]. Setting the value too low can lead to wasted bandwidth, while setting it too high can cause premature degradation of image quality.

[0060] S52. Based on the calculated dynamic bitrate weight For the total available target bitrate Distribute: ROI Target Bitrate : ; Background target bitrate : .

[0061] S53. Adjust the quantization parameters (QP) of the video encoder (such as H.264 / H.265 encoder): ROI region quantization parameters : ; Background area quantization parameters : ; In the formula, For ROI gradient energy, The gradient energy of the background region; These are the encoder feature coefficients, which can be obtained through... Calculation, where These are the encoder rate model parameters (preset by the H.264 / H.265 encoder kernel). For frame rate; To round down; This is a boundary truncation function, which restricts the calculation result to fall within the standard QP value range. For example, in this embodiment, the H.264 / H.265 standard is... .

[0062] S54. The controller sends instructions to the encoder to set the ROI macroblock. Setting background macroblocks Finally, the encoded video stream is sent to the user terminal via an IoT gateway.

[0063] In this embodiment 1, the adaptive weight allocation strategy establishes a differentiated bitrate weight allocation mechanism by jointly analyzing the semantic importance factor and the current channel transmission resistance coefficient. By using gradient energy to guide the dynamic adjustment of the quantization parameter (QP), under the condition of limited total bandwidth, it actively sacrifices the image quality of low-frequency background areas and concentrates the bitstream resources on encoding high-frequency ROI areas. This solves the problem of key information loss caused by the one-size-fits-all compression of traditional ABR technology, and ensures that the visitor's facial features remain clear and sharp when the network generates moderate resistance, thus achieving the optimal balance between visual experience and the current network status.

[0064] S6. Implement the background freeze strategy, which includes the following steps: Transmitter Operation: The image processing unit extracts the ROI region using a semantic mask and encodes only this irregular region at an extremely low bitrate. Simultaneously, it generates a reconstruction instruction packet {Type:Reconstruct,Ref_ID: X, ROI_Payload:...} and sends it to the receiver. This mode significantly reduces the amount of data transmitted.

[0065] Receiver operation: The receiver parses the instructions and retrieves the corresponding high-definition background reference frame from the local buffer. The received dynamic ROI regions are overlaid and blended in real time through coordinate mapping. The image is then rendered by feathering and smoothing the edges to create a composite image of a static high-definition background and a dynamic, clear human face.

[0066] Furthermore, during the background freezing strategy, to prevent lighting faults or background artifacts in the synthesized image at the receiving end due to gradual changes in ambient lighting (such as brightness decay caused by sunset) or changes in scene structure (such as temporary occlusion left by passersby), a semantically aware background spatiotemporal consistency update mechanism is introduced. This mechanism is embedded in the dual-modal transmission control module, and the specific implementation steps are as follows: S61. Real-time background validity monitoring: At the sending end, even in a non-congested state, the system will periodically (e.g., every...) (seconds) Compare the non-ROI region in the currently acquired frame with the currently stored background reference frame. To quantify the degree of background degradation, a background aging index is calculated.

[0067] S62. In order to balance the consistency of illumination and the stability of physical structure, the background aging index... The calculation formula is designed as follows: ; In the formula, for Background aging index at any given time, range of values ; This is the average brightness value of the background region (the region after removing the ROI) in the current video frame; The average brightness value of the currently active background reference frame; To prevent small constants with a denominator of zero (take...) ); For the current moment The video frames; This is a structural similarity index calculated only for the background region, used to detect structural changes caused by object displacement or non-illumination. These are the light-sensitive weights and the structure-sensitive weights, respectively.

[0068] For example, in this embodiment 1, the light sensitivity weight With structurally sensitive weights The method for determining it is as follows: Step 1, before the video call is established Seconds (e.g., the previous second, i.e.) (Frame), assuming the screen has a clean background (or is forcibly locked and no dynamic stream is transmitted), the system captures... For each frame of the image, calculate the relative frame of each frame to the reference frame. Light change value and structural difference value : ; The second step is to calculate the standard deviation of the two sequences to quantify the illumination noise and structural noise of the camera under the current environment: ; ; In the formula, for The arithmetic mean; for The arithmetic mean.

[0069] Step 3, to ensure It exhibits equal probability sensitivity to both types of changes and can automatically suppress interference from noisy terms, using the reciprocal of the standard deviation for normalization. ; .

[0070] S63, Set aging threshold - and aging threshold At the same time, based on the calculated Implement a tiered update strategy to avoid mistakenly updating images containing dynamic distractions (such as pedestrians passing by) as background elements: Clean frame verification: Only when the semantic analysis module determines that the current frame does not contain any high-confidence regions of interest (i.e., The background update process is only allowed to be triggered when a pedestrian or vehicle passes by. This ensures that passing pedestrians or vehicles are not "fixed" into the background.

[0071] Differential update execution: like This means that the current background difference is within the allowable range, and the system directly uses the locally cached high-definition background reference frame. ; like In other words, if only brightness shift occurs while the structure remains unchanged, the system does not retransmit the entire frame. Instead, it extracts the brightness histogram parameters of the current background and generates a lightweight "brightness correction command packet" to send to the receiving end. The receiving end then uses this packet to adjust its local cache. Perform global brightness mapping adjustment; like This means that the background changes significantly, such as turning on a light or moving an object, and the current network channel transmission resistance coefficient is high. (Network conditions are acceptable) The system immediately encodes the current clean frame into a new IDR frame, marks it with "Update_Ref_ID" and sends it, forcibly refreshing the receiver's buffer.

[0072] For example, in this embodiment 1, the aging threshold is... and aging threshold The determination method is as follows: during the silent calibration period of system initialization (and calculation) (time period synchronization of values), the system is based on the determined time period. Actual calculation Background aging index sequence of frame images And calculate the sample mean of the sequence. with sample standard deviation Using statistically common methods Criteria for defining the physical boundary between noise and significant change: ; .

[0073] S64. After receiving a new background reference frame or correction instruction, the receiving end does not directly perform a hard cut, but instead uses double-buffered blending rendering. During the background transition blending frame time, the transparency coefficient is linearly changed to smoothly transition the old background to the new background, thereby eliminating the visual jump caused by abrupt changes in the reference frame.

[0074] In this embodiment 1, the background freezing strategy serves as an emergency response mechanism when the network is in a state of extreme congestion. Unlike conventional video streaming techniques that transmit complete frames, this strategy employs a rendering strategy that combines dynamic ROI incremental transmission with local static background reuse. By encoding only key semantic regions with extremely low data volume at the sending end and reconstructing the image using cached reference frames at the receiving end, it solves the communication interruption problem caused by insufficient bandwidth in extremely weak network environments. This ensures that even when the channel capacity is close to its physical limit, the visitor's face can still be clearly presented with minimal traffic cost, thus guaranteeing the continuity of the core function of identity verification.

[0075] Example 2 As a second embodiment of the present invention, based on embodiment 1, a building video intercom terminal is also disclosed, such as... Figure 3As shown, the system specifically includes: a wide-angle camera module, a high-definition display screen, an audio codec, an IoT communication module, a main controller, and the IoT-enabled building visualization intercom system described in Embodiment 1 above. The main controller operates the IoT-enabled building visualization intercom system and is electrically connected to the wide-angle camera module, the high-definition display screen, and the IoT communication module. The wide-angle camera module is used to collect visitor video streams, the IoT communication module is used to monitor network channel status and transmit encoded data packets, and the high-definition display screen is used to display reconstructed or decoded video images on the resident's end.

[0076] During the video call setup phase, when a visitor presses the call button at the unit door station, the wide-angle camera module starts and captures images at 25fps. The community building visualization intercom system, utilizing IoT technology, first controls the main controller to perform histogram equalization preprocessing on the original image, converting the image to YCrCb space and adaptively enhancing the luminance component to eliminate the blackening of faces caused by backlighting.

[0077] During the video stream transmission and control phase, the IoT communication module detects network jitter in real time. ) and round-trip delay ( The system constructs a channel transmission resistance model based on physical parameters and objectively calculates the trend sensitivity coefficient by combining the receiver buffer capacity. Predict the channel transmission resistance coefficient at the next moment. .

[0078] Scenario 1: Network Fluctuation Period: Channel transmission resistance coefficient at the next moment Within a controllable fluctuation range, i.e. High-risk resistance threshold At this time, the system executes an adaptive weight allocation strategy. The main controller uses a lightweight convolutional neural network to locate the visitor's face (ROI region) and calculates the semantic importance factor based on the relative information density. Subsequently, the system calculates the weight adjustment sensitivity factor based on the encoder's physical characteristics. The quantization parameters (QP) of the ROI and background areas are dynamically adjusted. The IoT communication module prioritizes sending data packets carrying high-resolution facial features to ensure that the high-definition display screen at the resident's end can still clearly display the visitor's facial details when bandwidth is limited, while the background area is allowed to be appropriately blurred.

[0079] Scenario 2: Network congestion period: For example, when the signal is extremely weak in a basement, the predicted channel transmission resistance coefficient... Breaking through high-risk resistance threshold The system automatically switches to a background freeze strategy. The main controller encodes only the ROI region at an extremely low bitrate and generates reconstruction instructions; the IoT communication module only sends tiny ROI data packets and instructions. After receiving the data, the resident uses the locally cached high-definition background reference frame and the received dynamic ROI for real-time fusion rendering, displaying a smooth composite image of "static background plus dynamic face" on a high-definition display screen, thereby ensuring uninterrupted communication connection and availability of core identity verification functions even in extreme network environments.

[0080] Furthermore, a lightweight feature extraction network based on inverse residual structure and channel attention mechanism is constructed for frame-by-frame inference of video frames, specifically including: a. Input layer dimensionality reduction and preprocessing: The captured video frames Adjust to standard input size (For example The algorithm is then normalized. To reduce the computational cost of the first layer, a step size of 2 is used. Standard convolutional layers perform primary feature extraction, rapidly downsampling the feature map size to... The number of output channels is expanded to (e.g., 16 channels).

[0081] b. Construct depthwise separable convolutional (DSC) feature extraction units: The backbone of the network consists of several cascaded inverted residual blocks. For the first... For each feature layer, depthwise separable convolutions are used instead of standard convolutions. The specific operation is as follows: Dimensional Upgrading by Nodes: Utilizing The convolution kernel will take the number of input channels Expand to (Expansion multiplier) Typically, a value of 6 is used to enrich feature information using high-dimensional space; Depthwise convolution: utilizing or The convolutional kernel performs spatial convolution on each channel after expansion, with the stride set to 1 or 2 depending on the network depth. Point-by-point dimensionality reduction: reuse The convolution kernel compresses the feature channels back. Linear activation is employed to preserve the integrity of the feature manifold.

[0082] c. Embed the SE channel attention module: After the depthwise convolution of the aforementioned inverted residual blocks, a lightweight SE attention module is embedded to explicitly enhance semantic features.

[0083] Compression: Global average pooling is performed on the feature map to reduce the spatial dimension. Compress to Channel descriptor; Incentive: The correlation between channels is learned through two fully connected (FC) layers. The first FC layer compresses the channels into... The second fully connected (FC) layer restores the number of channels and generates channel weight vectors through a Hard-Sigmoid activation function. ; Recalibration: Recalibrating the weight vector Multiply by the corresponding channel of the original feature map.

[0084] d. Apply the Hard-Swish hardware-friendly activation function: To avoid complex exponential operations performed by traditional Sigmoid or Tanh functions on embedded devices, all non-linear activation layers in the network use the Hard-Swish function: ; In the formula, This represents the feature map data input to the activation function.

[0085] e. Multi-scale feature fusion and SSDLite detection head output: Feature maps of different scales are derived from layers 7 (downsampled 8 times), 13 (downsampled 16 times), and the final layer (downsampled 32 times) of the network backbone. The SSDLite (Separable SingleShotDetector) prediction head is used, and depthwise separable convolutions are employed to regress the bounding box coordinates of the Regions of Interest (ROIs) onto these feature maps. and category confidence .

[0086] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A community building visual intercom system utilizing Internet of Things (IoT) technology, characterized in that, include: The image acquisition and preprocessing module is used to build a multi-dimensional data acquisition mechanism with time-series memory capabilities, capture video streams, and perform histogram equalization preprocessing on the images. The communication module detects network parameters via the RTCP protocol; it extracts a frame without moving targets from the image as a high-definition background reference frame. The semantic analysis and quantization module is used to perform semantic segmentation on video frames, identify regions of interest and background regions, and calculate semantic importance factors based on the relative information density of regions of interest to quantify the value distribution of the image content. The channel state prediction module is used to construct a channel transmission resistance model based on network parameters, calculate the current channel transmission resistance coefficient, and predict the channel transmission resistance coefficient at the next moment based on the changing trend of the resistance coefficient. The dual-mode transmission control module is used to set a high-risk resistance threshold and compare the channel transmission resistance coefficient at the next moment with the high-risk resistance threshold. If the predicted coefficient is less than or equal to the threshold, an adaptive weight allocation strategy is executed. If the prediction coefficient is greater than the threshold, the background freezing strategy is executed.

2. The community building visual intercom system utilizing Internet of Things (IoT) technology according to claim 1, characterized in that, The histogram equalization preprocessing steps include: converting the original image from the RGB color space to the YCrCb color space and extracting the luminance component. And divided into Divide the data into non-overlapping rectangular sub-blocks; calculate the grayscale histogram of each sub-block and set a clipping threshold. For gray levels with excessive frequency, the number of clipped pixels is evenly distributed across all gray levels in the histogram of that sub-block to obtain a smoothed histogram. The cumulative distribution function of each sub-block is calculated to establish a local gray-level mapping table. Bilinear interpolation is used to remap the pixels to obtain the corrected brightness value. Finally, the corrected brightness value is merged with the original chroma component and sent to the encoder in YUV format.

3. The community building visual intercom system utilizing Internet of Things (IoT) technology according to claim 1, characterized in that, The formula for calculating the semantic importance factor is as follows: ； In the formula, As a semantic importance factor, This is a dynamic safety net factor. The area percentage of the region of interest. This represents the confidence level based on the output of a lightweight convolutional neural network.

4. The community building visual intercom system utilizing Internet of Things technology according to claim 3, characterized in that, The formula for calculating the channel transmission resistance coefficient is as follows: ； In the formula, For the current moment The channel transmission resistance coefficient, For a moment Network jitter value, For a moment Round-trip delay, To the maximum tolerable delay, To dilute the weight, For delay weighting, This is the sensitivity adjustment index.

5. The community building visual intercom system utilizing Internet of Things technology according to claim 4, characterized in that, The formula for calculating the channel transmission resistance coefficient at the next moment is: ； In the formula, This represents the channel transmission resistance coefficient at the next time step. For the current moment The channel transmission resistance coefficient, For time intervals, The rate of change of the channel transmission resistance coefficient. This is the trend sensitivity coefficient; The trend sensitivity coefficient The calculation formula is: ,in, For video capture frame rate, Configure the buffer capacity for the receiving terminal.

6. The community building visual intercom system utilizing Internet of Things (IoT) technology according to claim 1, characterized in that, The method for determining the high-risk resistance threshold is as follows: The physical critical state of network transmission is defined as the sum of weighted network jitter and round-trip delay equaling the maximum tolerable delay constant; at this point, the normalized network pressure value is 1. The high-risk resistance threshold is then calculated. ,in This is the high-risk resistance threshold.

7. The community building visual intercom system utilizing Internet of Things technology according to claim 4, characterized in that, The specific steps of the adaptive weight allocation strategy include: based on semantic importance factors and channel transmission resistance coefficient Calculate the dynamic bitrate weight of the region of interest ; Allocate ROI target bitrate based on dynamic bitrate weights and background target bitrate Based on the gradient energy of the ROI region and background gradient energy Calculate and adjust the quantization parameters of the ROI region respectively. and background area quantization parameters The calculation of the quantization parameters introduces a boundary truncation function to limit the values ​​to the range of the standard QP.

8. The community building visual intercom system utilizing Internet of Things technology according to claim 7, characterized in that, The dynamic bitrate weight of the region of interest The calculation formula is: ； in, The background suppression index is used.

9. The community building visual intercom system utilizing Internet of Things (IoT) technology according to claim 1, characterized in that, The specific steps of the background freezing strategy include: at the sending end, extracting the region of interest using semantic masking and performing low bitrate encoding to generate an instruction packet containing reconstruction instructions and region of interest data, which is then sent to the receiving end; at the receiving end, parsing the instruction packet, retrieving the pre-stored high-definition background reference frame from the local cache, and real-time superimposing and merging the received dynamic region of interest onto the high-definition background reference frame through coordinate mapping, and performing feathering smoothing on the edges to render the composite image.

10. A building video intercom terminal, characterized in that, Specifically, it includes: The system comprises a wide-angle camera module, a high-definition display screen, an IoT communication module, a main controller, and a community building visualization intercom system utilizing IoT technology as described in any one of claims 1-9; the main controller is connected to the wide-angle camera module, the high-definition display screen, and the IoT communication module respectively, and operates the community building visualization intercom system utilizing IoT technology, controlling the wide-angle camera module to collect visitor video streams; the IoT communication module is used to monitor network channel status and transmit encoded data packets, and the high-definition display screen is used to display reconstructed or decoded video images at the resident's end.

Images