Building equipment operation state evaluation method and system based on fusion of machine vision and Internet of Things

Abstract

The invention relates to the technical field of data processing, and discloses a machine vision and Internet of Things fused building equipment operation state evaluation method and system, and the method comprises the steps: carrying out the weight distribution of the image feature data of building equipment, and obtaining the weighted image feature data; performing element-by-element fusion on the weighted image feature data and standardized parameters to obtain cross-modal representation of the building equipment; performing dynamic correlation analysis on mechanical operation characteristics and electrical operation characteristics in the cross-modal representation to obtain a state coupling characteristic set of the building equipment; mapping the state coupling feature set into a comprehensive health degree score of the building equipment; outputting a health status report and a maintenance suggestion of the building equipment based on the comprehensive health degree score; according to the invention, the accuracy of building equipment operation state evaluation based on machine vision and Internet of Things fusion can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a building equipment operation state evaluation method and system based on machine vision and Internet of Things. BACKGROUND

[0002] In the field of building equipment operation state evaluation, existing technologies mostly rely on single data source for analysis, which is difficult to comprehensively reflect the real operation state of the equipment. When only relying on image data obtained by machine vision, the appearance features of the equipment can be captured, but the running details of the internal mechanical and electrical systems cannot be known in depth, resulting in one-sidedness in the judgment of potential faults of the equipment. While simply relying on parameter data collected by Internet of Things, the intuitive perception of the external morphological changes of the equipment is lacking, so that the evaluation results are likely to ignore the chain problems caused by appearance defects.

[0003] At the same time, existing technologies have obvious deficiencies in data fusion and feature analysis. Different sources of data are often difficult to effectively integrate due to large differences in dimensions and formats, resulting in the correlation of cross-modal information being fragmented. The correlation analysis of mechanical operation features and electrical operation features is mostly static, which cannot dynamically capture the coupling relationship between the two over time, and the sensitivity to the operation state of the equipment is insufficient, which makes it difficult to accurately identify early fault signals, resulting in low evaluation accuracy and providing no reliable basis for equipment maintenance. SUMMARY

[0004] The present application provides a building equipment operation state evaluation method and system based on machine vision and Internet of Things, which mainly aims to solve the problems raised in the background technology.

[0005] To achieve the above-mentioned purpose, the present application provides a building equipment operation state evaluation method based on machine vision and Internet of Things, comprising: S1. Weight distribution is performed on image feature data of the building equipment to obtain weighted image feature data; S2. The weighted image feature data is element-wise fused with standardized parameters to obtain cross-modal representation of the building equipment; S3. Dynamic correlation analysis is performed on the mechanical operation features and electrical operation features in the cross-modal representation to obtain a state coupling feature set of the building equipment; S4. The state coupling feature set is mapped to a comprehensive health degree score of the building equipment; S5. A health state report and maintenance suggestion of the building equipment are output based on the comprehensive health degree score.

[0006] In a preferred embodiment, the weight distribution on the image feature data of the building equipment to obtain the weighted image feature data comprises: perform spatial feature decoupling on the image feature data to obtain a key appearance feature region of the construction equipment; perform defect risk analysis on the key appearance feature region to obtain a preliminary risk weight value of the construction equipment; perform multi-modal collaborative optimization on the preliminary risk weight value to obtain a dynamic weight coefficient of the construction equipment; perform dynamic weighted fusion on the image feature data based on the dynamic weight coefficient to obtain weighted image feature data.

[0007] In a preferred embodiment, the step of performing element-by-element fusion of the weighted image feature data and the standardized parameter includes: perform dimension alignment on the weighted image feature data to obtain a dimension-matched standardized parameter matrix; perform tensor fusion on the weighted image feature data and the standardized parameter matrix to obtain a primary fusion feature matrix of the construction equipment; perform cross-modal feature conversion on the primary fusion feature matrix to obtain an enhanced cross-modal intermediate feature.

[0008] In a preferred embodiment, the step of performing cross-modal feature conversion on the primary fusion feature matrix to obtain an enhanced cross-modal intermediate feature includes: perform channel-level feature fusion on each channel in the cross-modal intermediate feature to obtain a dynamic health state vector of the construction equipment; construct a channel attention weight matrix based on the dynamic health state vector; perform attention weighted fusion on the cross-modal intermediate feature based on the channel attention weight matrix to obtain an optimized cross-modal representation; perform layer normalization processing on the optimized cross-modal feature representation to obtain a cross-modal representation of the construction equipment.

[0009] In a preferred embodiment, the step of performing dynamic correlation analysis on the mechanical operation feature and the electrical operation feature in the cross-modal representation to obtain a state coupling feature set of the construction equipment includes: perform time domain alignment on the current fluctuation sequence in the electrical operation feature to obtain an aligned current fluctuation sequence; perform dynamic correlation degree calculation on the aligned current fluctuation sequence and the vibration amplitude sequence based on a coupling coefficient formula to obtain a state coupling feature set of the construction equipment, wherein the coupling coefficient formula is:

[0010] wherein, For dynamic correlation, It is a sequence of vibration amplitudes. It is a current fluctuation sequence. This represents the instantaneous change in the vibration amplitude sequence. This represents the instantaneous change in the current fluctuation sequence. The standard deviation of the vibration amplitude sequence. denoted as the standard deviation of the current fluctuation sequence.

[0011] In a preferred embodiment, the step of time-domain aligning the current fluctuation sequence in the electrical operating characteristics to obtain the aligned current fluctuation sequence includes: Phase offset compensation is performed on the reference time axis of the current fluctuation sequence to obtain the phase offset compensated current fluctuation sequence; The current fluctuation sequence after phase offset compensation is normalized to obtain a time-domain aligned current fluctuation sequence.

[0012] In a preferred embodiment, mapping the state-coupled feature set to a comprehensive health score of the building equipment includes: The dynamic health indicators of the building equipment are obtained by weighted fusion of the multi-source dynamic characteristics in the state coupling feature set. The dynamic health indicators are reduced by a vital sign scaling method to obtain a multi-level health score for the building equipment. The multi-level health scores are aggregated to obtain the comprehensive health score of the building equipment.

[0013] In a preferred embodiment, the step of performing vital sign scaling reduction on the dynamic health indicators to obtain a multi-level health score for the building equipment includes: When the dynamic health indicator is in a stable range, a high health score is output. When the dynamic health indicator is in the transition range, a medium health score is output. When the dynamic health indicator is in an abnormal range, a low health score is output.

[0014] In a preferred embodiment, the step of outputting a health status report and maintenance recommendations for the building equipment based on the comprehensive health score includes: The comprehensive health score is matched with a preset health reference value, and the matching result is encoded into a corresponding report generation instruction. Based on the report generation instructions, a health status report and maintenance recommendations for the building equipment are generated.

[0015] To address the aforementioned problems, the present invention also provides a building equipment operation status assessment system integrating machine vision and the Internet of Things, the system comprising: The image feature data acquisition module is used to assign weights to the image feature data of building equipment to obtain weighted image feature data. The cross-modal fusion module is used to fuse the weighted image feature data with the standardized parameters element by element to obtain the cross-modal representation of the building equipment; The dynamic correlation analysis module is used to perform dynamic correlation analysis on the mechanical operation characteristics and electrical operation characteristics in the cross-modal representation to obtain the state coupling feature set of the building equipment. A health rating module is used to map the state coupling feature set to a comprehensive health rating of the building equipment; The report output module is used to output a health status report and maintenance recommendations for the building equipment based on the comprehensive health score.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention dynamically weights the image feature data of building equipment, combines cross-modal fusion technology to deeply fuse the weighted image feature data with standardized parameters, generates a comprehensive cross-modal representation, and then uses dynamic correlation analysis to explore the intrinsic relationship between mechanical operation features and electrical operation features to obtain a state coupling feature set, which is finally mapped to a comprehensive health score. This series of processes significantly improves the accuracy of building equipment operation status assessment and can more accurately reflect the actual health status of the equipment.

[0017] 2. Meanwhile, this technology achieves a refined assessment of equipment status through multimodal collaborative optimization of weights, channel attention mechanism to enhance cross-modal features, and hierarchical health scoring. The health status reports and maintenance recommendations generated based on the assessment results are more targeted and practical, effectively improving the efficiency and reliability of building equipment operation status assessment and providing a scientific basis for equipment maintenance. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a method for evaluating the operational status of building equipment that integrates machine vision and the Internet of Things, provided in an embodiment of the present invention. Figure 2 A functional block diagram of a building equipment operation status assessment system integrating machine vision and the Internet of Things provided in an embodiment of the present invention; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] This application provides a method for assessing the operational status of building equipment that integrates machine vision and the Internet of Things (IoT). The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method for assessing the operational status of building equipment that integrates machine vision and IoT can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0021] Reference Figure 1 The diagram shown is a flowchart illustrating a method for assessing the operational status of building equipment that integrates machine vision and the Internet of Things (IoT) according to an embodiment of the present invention. In this embodiment, the method for assessing the operational status of building equipment that integrates machine vision and IoT includes: S1. Weight the image feature data of building equipment to obtain weighted image feature data; In this embodiment of the invention, the weighting of the image feature data of the building equipment to obtain weighted image feature data includes: Spatial feature decoupling is performed on the image feature data to obtain the key appearance feature regions of the building equipment; Defect risk analysis is performed on the key appearance feature areas to obtain the preliminary risk weight of the building equipment; Multimodal collaborative optimization is performed on the preliminary risk weights to obtain the dynamic weight coefficients of the building equipment; The image feature data is dynamically weighted and fused based on the dynamic weight coefficients to obtain weighted image feature data.

[0022] Specifically, spatial feature decoupling is performed on the image feature data of building equipment. This is achieved by dividing the image feature data into different spatial regions using image segmentation technology. Each region corresponds to a part of the building equipment's appearance. Then, feature extraction tools are used to identify regions that play a key role in evaluating the equipment's function and status. These regions together constitute the key appearance feature regions of the building equipment.

[0023] Furthermore, defect risk analysis of key appearance feature areas involves comparing each key appearance feature area with the defect features in the database of common defects of this type of building equipment one by one, and counting the number of times and the degree of matching between each area and the defect features.

[0024] Furthermore, based on the matching results, the probability of defects appearing in each key appearance feature area is determined, and this probability is converted into a specific numerical value, which is the initial risk threshold of the building equipment.

[0025] Furthermore, multimodal collaborative optimization of the initial risk weights involves combining various information such as the building equipment's historical operating data, maintenance records, and environmental impact factors.

[0026] Furthermore, this information is correlated with the initial risk weights. For the parts where the initial risk weights deviate from the actual situation, adjustments are made based on the correlation analysis results. This ensures that the adjusted weights can reflect the defect risks of key appearance feature areas and adapt to different operating and environmental conditions. The resulting adjusted values ​​are the dynamic weight coefficients of the building equipment.

[0027] Furthermore, dynamic weighted fusion of image feature data based on dynamic weight coefficients involves multiplying the value corresponding to each feature point in the image feature data by the dynamic weight coefficient of the key appearance feature region where that feature point is located.

[0028] Furthermore, the results of all multiplications are summarized and integrated to form a new set of image data that highlights important features. This set of data is the weighted image feature data.

[0029] In summary, spatial feature decoupling of image feature data to obtain key appearance feature regions can accurately locate the appearance parts of building equipment that play an important role in assessing its operating status, eliminate interference from irrelevant areas, and make subsequent analysis more focused on core features, laying the foundation for accurate assessment.

[0030] In summary, by conducting defect risk analysis on key appearance feature areas, preliminary risk weights can be obtained, which can quantify the probability of defects occurring in different key areas, giving a clear numerical representation of the risk level of each area, and facilitating subsequent targeted handling.

[0031] In summary, multimodal collaborative optimization of the initial risk weights yields dynamic weight coefficients. By combining various aspects of equipment information to adjust the weights, the weights can not only reflect defect risks but also adapt to different operating conditions of the equipment, thereby improving the accuracy and adaptability of the weights.

[0032] In summary, the weighted image feature data obtained by dynamically weighting and fusing image feature data based on dynamic weight coefficients can highlight the influence of important features and weaken secondary features, making the fused image feature data more realistically reflect the equipment status and thus improving the accuracy of building equipment operation status assessment.

[0033] S2. The weighted image feature data is fused with the standardized parameters element by element to obtain the cross-modal characterization of the building equipment; In this embodiment of the invention, the step of fusing the weighted image feature data with the standardized parameters element-by-element includes: Based on the weighted image feature data, the standardized parameters are dimension-aligned to obtain a dimension-matched standardized parameter matrix; Tensor fusion is performed on the weighted image feature data and the standardized parameter matrix to obtain the primary fusion feature matrix of the building equipment; The primary fusion feature matrix is ​​subjected to cross-modal feature transformation to obtain enhanced cross-modal intermediate features.

[0034] The step of performing cross-modal feature transformation on the primary fused feature matrix to obtain enhanced cross-modal intermediate features includes: Channel-level feature fusion is performed on each channel in the cross-modal intermediate features to obtain the dynamic health status vector of the building equipment; Construct a channel attention weight matrix based on the dynamic health status vector; Based on the channel attention weight matrix, the cross-modal intermediate features are subjected to attention-weighted fusion to obtain the optimized cross-modal representation; The optimized cross-modal feature representation is subjected to layer normalization to obtain the cross-modal characterization of the building equipment.

[0035] Specifically, the standardized parameters are dimensionally aligned based on the weighted image feature data. First, the number of dimensions of the weighted image feature data is determined, and then the number of dimensions of the standardized parameters is checked.

[0036] Furthermore, if the number of dimensions of the standardized parameters is less than the number of dimensions of the weighted image feature data, then null parameters corresponding to the extra dimensions in the weighted image feature data are added to the standardized parameters.

[0037] Furthermore, if the number of dimensions of the standardized parameters exceeds the number of dimensions of the weighted image feature data, then the redundant parameters in the standardized parameters that exceed the number of dimensions of the weighted image feature data are deleted. After this adjustment, a standardized parameter matrix with dimension matching that is consistent with the dimensions of the weighted image feature data is obtained.

[0038] Furthermore, tensor fusion is performed on the weighted image feature data and the standardized parameter matrix of dimension matching, treating the weighted image feature data as a three-dimensional array, where the first dimension represents the pixels in the height direction of the image, the second dimension represents the pixels in the width direction of the image, and the third dimension represents the feature channels of the image.

[0039] Furthermore, the standardized parameter matrix for dimension matching is also adjusted to a three-dimensional array with the same structure. Then, the elements at corresponding positions in the two three-dimensional arrays are multiplied to obtain a new three-dimensional array, which is the primary fusion feature matrix of the building equipment.

[0040] Furthermore, cross-modal feature transformation is performed on the primary fusion feature matrix, first identifying features belonging to the image modality and features belonging to the parameter modality in the primary fusion feature matrix.

[0041] Furthermore, a mapping relationship between the two modal features is established. That is, based on the numerical range and trend of the image modal features, the values ​​of the parameter modal features are adjusted to a suitable range, while retaining the core information of the two modal features.

[0042] Furthermore, through such transformation processing, an enhanced cross-modal intermediate feature is obtained that contains both image feature information and parametric feature information, and the two types of information are mutually adapted.

[0043] Specifically, channel-level feature fusion is performed on each channel in the cross-modal intermediate features. First, all channels included in the cross-modal intermediate features are determined, and each channel corresponds to feature information of different aspects of the building equipment. Then, the values ​​of all elements in each channel are extracted one by one.

[0044] Furthermore, the average value of all elements in each channel is calculated, and the average values ​​of all channels are arranged in channel order to form a vector containing the average characteristic information of each channel. This vector is the dynamic health status vector of the building equipment.

[0045] Furthermore, a channel attention weight matrix is ​​constructed based on the dynamic health status vector. First, the magnitude of each value in the dynamic health status vector is counted. The larger the value, the more important the characteristics of the corresponding channel are to the health status assessment of building equipment.

[0046] Furthermore, the attention weight of the corresponding channel is determined according to the proportion of each value in the vector. These weights are arranged in channel order into a matrix form with the same number of intermediate feature channels across modalities. This matrix is ​​the channel attention weight matrix.

[0047] Furthermore, attention-weighted fusion of cross-modal intermediate features is performed based on the channel attention weight matrix. The weight value of each channel in the channel attention weight matrix is ​​multiplied by all elements of the corresponding channel in the cross-modal intermediate features, so that the feature values ​​of important channels are amplified and the feature values ​​of secondary channels are suppressed. Then, the elements of all channels after weighting are integrated together to form a new feature set. This new feature set is the optimized cross-modal representation.

[0048] Furthermore, layer normalization is performed on the optimized cross-modal feature representation. First, the average value and dispersion of all elements in the optimized cross-modal feature representation are calculated. Then, the average value of each element is subtracted from the average value and divided by the square root of the dispersion. This ensures that all element values ​​are within a uniform range and the overall distribution is more stable. The result obtained after this processing is the cross-modal characterization of building equipment.

[0049] In summary, dimensional alignment of standardized parameters based on weighted image feature data yields a dimension-matched standardized parameter matrix, which eliminates dimensional differences between the two types of data, ensures consistent data structure during subsequent fusion, avoids fusion deviations caused by dimensional mismatch, and provides a foundation for accurate fusion.

[0050] In summary, tensor fusion of weighted image feature data and dimensional matching standardized parameter matrix yields a primary fusion feature matrix. This allows for deep integration of image features and standardized parameters from multiple dimensions, consolidating the device status information carried by both types of data. As a result, the primary fusion feature matrix contains richer device features, providing comprehensive data support for further analysis.

[0051] In summary, performing cross-modal feature transformation on the primary fusion feature matrix to obtain enhanced cross-modal intermediate features can effectively transform and enhance image modal and parametric modal features, strengthen the correlation between the two modal features, improve the feature representation ability, and thus lay the foundation for obtaining accurate cross-modal representations and improving the accuracy of building equipment operation status assessment.

[0052] In summary, channel-level feature fusion of each channel in the cross-modal intermediate features yields a dynamic health status vector, which can integrate feature information from different channels, extract core features reflecting the overall health status of building equipment, provide concise and crucial status basis for subsequent analysis, and avoid the limitations of single-channel features.

[0053] In summary, constructing a channel attention weight matrix based on dynamic health status vectors allows for the allocation of weights according to the importance of each channel feature to the device health assessment, enabling more attention to the features of important channels in subsequent fusion and improving the relevance and effectiveness of the features.

[0054] In summary, the optimized cross-modal representation obtained by attention-weighted fusion of cross-modal intermediate features based on the channel attention weight matrix can highlight the influence of key features, suppress minor or interfering features, and make the fused representation more accurately reflect the device status and enhance the discriminative power of features.

[0055] In summary, performing layer normalization on the optimized cross-modal feature representation to obtain cross-modal characterization can make the feature data distribution more stable and uniform, eliminate the influence of differences in different feature scales, improve the accuracy of subsequent dynamic correlation analysis, and thus improve the overall accuracy of building equipment operation status assessment.

[0056] S3. Perform dynamic correlation analysis on the mechanical operation characteristics and electrical operation characteristics in the cross-modal characterization to obtain the state coupling characteristic set of the building equipment; In this embodiment of the invention, the step of performing dynamic correlation analysis on the mechanical operation characteristics and electrical operation characteristics in the cross-modal characterization to obtain the state coupling feature set of the building equipment includes: The current fluctuation sequence in the electrical operating characteristics is time-domain aligned to obtain the aligned current fluctuation sequence. Based on the coupling coefficient formula, the dynamic correlation between the aligned current fluctuation sequence and the vibration amplitude sequence is calculated to obtain the state coupling feature set of the building equipment, wherein the coupling coefficient formula is:

[0057] In the formula, For dynamic correlation, It is a sequence of vibration amplitudes. It is a current fluctuation sequence. This represents the instantaneous change in the vibration amplitude sequence. This represents the instantaneous change in the current fluctuation sequence. The standard deviation of the vibration amplitude sequence. denoted as the standard deviation of the current fluctuation sequence.

[0058] The step of performing time-domain alignment on the current fluctuation sequence in the electrical operating characteristics to obtain the aligned current fluctuation sequence includes: Phase offset compensation is performed on the reference time axis of the current fluctuation sequence to obtain the phase offset compensated current fluctuation sequence; The current fluctuation sequence after phase offset compensation is normalized to obtain a time-domain aligned current fluctuation sequence.

[0059] Specifically, the current fluctuation sequence in the electrical operation characteristics is time-domain aligned. First, the time start and time interval of the current fluctuation sequence are determined, and then the time start and time interval of the vibration amplitude sequence in the mechanical operation characteristics are obtained. The time start of the current fluctuation sequence is adjusted to be consistent with the time start of the vibration amplitude sequence.

[0060] Furthermore, the time interval of the current fluctuation sequence is unified to be the same as the time interval of the vibration amplitude sequence. For any missing time points that occur during the adjustment process, the average value of the current fluctuation values ​​of adjacent time points is used to fill the gaps. The current fluctuation sequence obtained after this processing is the aligned current fluctuation sequence.

[0061] Furthermore, based on the coupling coefficient formula, the dynamic correlation between the aligned current fluctuation sequence and the vibration amplitude sequence is calculated. The current fluctuation values ​​at each time point are extracted sequentially from the aligned current fluctuation sequence.

[0062] Furthermore, the vibration amplitude values ​​corresponding to the same time point are extracted from the vibration amplitude sequence. The current fluctuation value and the vibration amplitude value at the same time point are taken as a data pair. According to the calculation steps of the coupling coefficient formula, the sum of the products of the two values ​​in all data pairs is first calculated, and then the sum of squares of all values ​​in the current fluctuation sequence and the sum of squares of all values ​​in the vibration amplitude sequence are calculated respectively.

[0063] Furthermore, dividing the sum of the products by the square root of the product of the sum of the squares of the current fluctuation values ​​and the sum of the squares of the vibration amplitude values ​​yields the correlation degree between the two sequences within that time period. Arranging the correlation degrees of different time periods in chronological order forms the set of state coupling features of the building equipment.

[0064] Specifically, phase offset compensation is performed on the reference time axis of the current fluctuation sequence. First, the target time axis that is time-domain aligned with the current fluctuation sequence is determined, and then the time difference between the reference time axis and the target time axis of the current fluctuation sequence, i.e., the phase offset, is found.

[0065] Furthermore, the phase offset is added to or subtracted from the time value corresponding to each data point in the current fluctuation sequence to make the overall time distribution of the current fluctuation sequence consistent with the time distribution of the target time axis. The sequence obtained after such adjustment is the current fluctuation sequence after phase offset compensation.

[0066] Furthermore, the current fluctuation sequence after phase offset compensation is normalized. First, the time value range of all data points in the current fluctuation sequence after phase offset compensation is determined, namely the minimum time value and the maximum time value. Then, the time value range of the target time axis is determined, namely the target minimum time value and the target maximum time value.

[0067] Furthermore, the time value of each data point in the current fluctuation sequence after phase offset compensation is proportionally converted to the time value range of the target time axis.

[0068] Furthermore, the minimum time value of the sequence after phase offset compensation is subtracted from the time value of each data point, multiplied by the ratio of the time range of the target time axis to the time range of the sequence after phase offset compensation, and finally the target minimum time value is added. The sequence obtained after this transformation is the time-domain aligned current fluctuation sequence.

[0069] Specifically, in the coupling coefficient formula, the vibration amplitude sequence comes from the vibration sensor records of the mechanical operation of the building equipment, and is sorted by time; the current fluctuation sequence comes from the current sensor records of the electrical operation, and is sorted by time; the instantaneous change of the vibration amplitude sequence is the difference between the vibration amplitudes at adjacent moments; the instantaneous change of the current fluctuation sequence is the difference between the current values ​​at adjacent moments.

[0070] Furthermore, the standard deviation of the vibration amplitude sequence is calculated by first taking the average of all vibration amplitudes, then taking the average of the squares of the differences between each value and the average, and finally taking the square root.

[0071] Furthermore, the standard deviation of the current fluctuation sequence is calculated in the same way as that of the vibration amplitude sequence, except that the calculation object is the current value.

[0072] Furthermore, this formula is used to quantitatively reflect the dynamic correlation between the mechanical vibration state and the electrical current state of building equipment. The ratio of the product of their instantaneous changes to the product of their respective discreteness reflects the synchronicity and correlation of the changes.

[0073] Furthermore, when the instantaneous changes of the two are in the same direction, have large amplitudes, and are relatively small in dispersion, the dynamic correlation increases; when the changes are in opposite directions, have small amplitudes, or are relatively large in dispersion, the dynamic correlation decreases.

[0074] In summary, time-domain alignment of the current fluctuation sequence in electrical operation characteristics yields an aligned current fluctuation sequence, which eliminates the time-dimensional deviation between the current fluctuation sequence and the vibration amplitude sequence. This ensures that both are analyzed on the same time scale, providing a time-synchronized data foundation for subsequent correlation calculations and avoiding correlation analysis errors caused by time asynchrony.

[0075] In summary, the state coupling feature set is obtained by dynamically calculating the correlation between the aligned current fluctuation sequence and the vibration amplitude sequence based on the coupling coefficient formula. This can quantify the dynamic correlation between mechanical operation features and electrical operation features, clearly present the synergistic change relationship between the two features, and enable the state coupling feature set to comprehensively reflect the linkage state of the mechanical and electrical systems during the operation of building equipment. This provides a key basis for accurately assessing the overall operating status of the equipment, thereby improving the accuracy and reliability of the assessment of the operating status of building equipment.

[0076] In summary, phase offset compensation of the reference time axis of the current fluctuation sequence yields a phase offset compensated current fluctuation sequence, which can correct the time phase deviation between the current fluctuation sequence and the vibration amplitude sequence, ensuring that the two are consistent in terms of time starting point and change rhythm. This lays the foundation for subsequent time domain alignment and avoids distortion of feature correlation analysis caused by phase differences.

[0077] In summary, normalizing the current fluctuation sequence after phase offset compensation yields a time-domain aligned current fluctuation sequence. This allows the time range of the current fluctuation sequence to be uniformly adjusted to match the vibration amplitude sequence, eliminating the difference between the two in terms of time scale. This enables the two sequences to be directly compared and correlated in the same time dimension, further improving the accuracy of dynamic correlation analysis between mechanical and electrical operating characteristics. This provides a guarantee for generating a reliable state coupling feature set, thereby helping to improve the accuracy of building equipment operating status assessment.

[0078] S4. Map the state coupling feature set to a comprehensive health score of the building equipment; In this embodiment of the invention, mapping the state coupling feature set to a comprehensive health score of the building equipment includes: The dynamic health indicators of the building equipment are obtained by weighted fusion of the multi-source dynamic characteristics in the state coupling feature set. The dynamic health indicators are reduced by a vital sign scaling method to obtain a multi-level health score for the building equipment. The multi-level health scores are aggregated to obtain the comprehensive health score of the building equipment.

[0079] The process of scaling down the dynamic health indicators to obtain a multi-level health score for the building equipment includes: When the dynamic health indicator is in a stable range, a high health score is output. When the dynamic health indicator is in the transition range, a medium health score is output. When the dynamic health indicator is in an abnormal range, a low health score is output.

[0080] Specifically, a weighted fusion of multi-source dynamic characteristics in the state coupling feature set is performed. First, the importance of each characteristic in the multi-source dynamic characteristics to the health status assessment of building equipment is determined, and then a corresponding fixed weight value is assigned to each characteristic.

[0081] Furthermore, the value of each vital sign is multiplied by its respective weight value, and then all the results of multiplication are added together to obtain a value that comprehensively reflects the overall situation of multi-source dynamic vital signs. This value is the dynamic health index of building equipment.

[0082] Furthermore, the dynamic health indicators are reduced by a scaling factor, and three continuous and non-overlapping intervals are pre-defined: a stable interval, a transitional interval, and an abnormal interval. Each interval corresponds to a health level, and the values ​​of the dynamic health indicators are compared with these three intervals.

[0083] Furthermore, if the value falls within the stable range, a high health score is output; if it falls within the transition range, a medium health score is output; and if it falls within the abnormal range, a low health score is output. These scores together constitute the multi-level health score of the building equipment.

[0084] Furthermore, the multi-level health scores are aggregated, and all high, medium and low health scores are collected. The frequency of each score is counted, and the score with the most frequency is used as the main reference. If there are cases with the same frequency, the higher score is used. The final score is used as the comprehensive health score of the building equipment.

[0085] Specifically, when the dynamic health indicators are in a stable range, the specific range of the stable range is first determined. This range is defined based on the historical health data of the building equipment during normal operation, and includes the dynamic health indicator values ​​when all functions of the equipment are in good condition.

[0086] Furthermore, the current dynamic health indicator value is compared with the upper and lower limits of the stable range. If the value is greater than or equal to the lower limit of the stable range and less than or equal to the upper limit of the stable range, it is determined that it is in the stable range. At this time, the preset high health score is directly output, which corresponds to the level of good equipment operation status and extremely low failure risk.

[0087] Specifically, when the dynamic health index is in the transition range, the specific range of the transition range is defined. This range is between the stable range and the abnormal range, and includes the dynamic health index values ​​when the equipment operating status begins to fluctuate slightly but has not yet reached the level of failure.

[0088] Furthermore, the current dynamic health indicator value is compared with the upper and lower limits of the transition range. If the value is greater than the upper limit of the stable range and less than the lower limit of the abnormal range, it is determined to be in the transition range. At this time, a preset medium health score is output, which corresponds to the level where the equipment is basically operating normally and there is a potential minor risk that needs attention.

[0089] Specifically, when the dynamic health indicator is in the abnormal range, the specific range of the abnormal range is determined. This range is defined based on historical data when building equipment malfunctions or is close to malfunction, and includes the dynamic health indicator values ​​when the equipment function is obviously abnormal.

[0090] Furthermore, the current dynamic health indicator value is compared with the lower limit of the abnormal range. If the value is greater than or equal to the lower limit of the abnormal range, it is determined that it is in the abnormal range. At this time, a preset low health score is output, which corresponds to the level of abnormal equipment operation status, high fault risk and need to be dealt with in time.

[0091] In summary, weighted fusion of multi-source dynamic vital signs in the state coupling feature set yields dynamic health indicators. This approach integrates dynamic vital sign information from different sources and of different types, and through reasonable weight allocation, highlights the impact of key vital signs on the health status of the equipment, avoiding the one-sidedness of a single vital sign and enabling dynamic health indicators to comprehensively reflect the real-time health status of the equipment.

[0092] In summary, the multi-level health score is obtained by scaling dynamic health indicators, which can map continuous dynamic health indicators to different health level scores, making the health status of equipment more intuitive and clear, and facilitating subsequent targeted analysis of different health levels.

[0093] In summary, the comprehensive health score obtained by aggregating the multi-level health scores can integrate the results of the multi-level scores, eliminate the random errors that may exist in a single score, and form a unified and comprehensive score result. This score can accurately summarize the overall health level of the equipment, provide a reliable basis for the subsequent output of health status reports and maintenance recommendations, and thus improve the scientificity and effectiveness of building equipment operation status assessment.

[0094] In general, a high health score is output when the dynamic health index is in a stable range, a medium health score is output when it is in a transitional range, and a low health score is output when it is in an abnormal range. This interval-based scoring method can transform continuous dynamic health indicators into discrete and intuitive health level scores, so that the health status of building equipment can be clearly quantified.

[0095] In summary, by presetting different intervals corresponding to different scores, it is possible to accurately distinguish between good operating conditions, potential risk conditions, and abnormal conditions of equipment, thus avoiding ambiguity in health status assessment.

[0096] In summary, multi-level health scores provide clear and hierarchical basic data for subsequent aggregation of levels to obtain a comprehensive health score, ensuring that the comprehensive health score accurately reflects the health status of equipment at each stage. This provides a reliable basis for generating targeted health status reports and maintenance recommendations, improving the accuracy and practicality of building equipment operation status assessment.

[0097] S5. Based on the comprehensive health score, output a health status report and maintenance recommendations for the building equipment.

[0098] In this embodiment of the invention, the step of outputting a health status report and maintenance recommendations for the building equipment based on the comprehensive health score includes: The comprehensive health score is matched with a preset health reference value, and the matching result is encoded into a corresponding report generation instruction. Based on the report generation instructions, a health status report and maintenance recommendations for the building equipment are generated.

[0099] Specifically, the overall health score is matched with preset health reference values, which are divided into multiple consecutive intervals. Each interval corresponds to a health status level. The overall health score is compared with these intervals to determine its corresponding health status level.

[0100] Furthermore, based on the health status level, the corresponding code is retrieved from the preset instruction library. This code contains information such as the basic format and content modules required to generate the report, and this code is the corresponding report generation instruction.

[0101] Furthermore, based on the report generation instructions, a health status report and maintenance recommendations for the building equipment are generated. Following the basic format in the report generation instructions, the comprehensive health score, the corresponding health status level, and key characteristic data during equipment operation are filled in to form a health status report.

[0102] Furthermore, based on the preset maintenance plan library corresponding to the health status level, suitable maintenance items, maintenance cycles, and specific operation steps are extracted as maintenance suggestions, and the health status report and maintenance suggestions are integrated into complete output content.

[0103] In summary, matching the comprehensive health score with preset health reference values ​​and encoding them into report generation instructions allows the system to accurately locate the report generation rules corresponding to the health status of building equipment, avoiding the subjectivity and errors of human judgment, ensuring that the report generation instructions strictly correspond to the actual health status of the equipment, and laying the foundation for generating accurate health status reports and maintenance recommendations in the future.

[0104] In summary, generating health status reports and maintenance recommendations based on report generation instructions ensures that the output content strictly adheres to the format and content requirements that match the comprehensive health score. This guarantees that the health status report accurately reflects the comprehensive health score and related status information of the equipment, and that the maintenance recommendations closely align with the actual health needs of the equipment. This enhances the relevance and reliability of the reports and recommendations, helps users efficiently understand the equipment status and take appropriate maintenance measures, and ultimately improves the practicality and application value of building equipment operation status assessment.

[0105] like Figure 2 The diagram shown is a functional block diagram of a building equipment operation status assessment system that integrates machine vision and the Internet of Things, provided by an embodiment of the present invention.

[0106] The building equipment operation status assessment system 100 integrating machine vision and the Internet of Things (IoT) of this invention can be installed in an electronic device. Depending on the functions implemented, the building equipment operation status assessment system 100 may include an image feature data acquisition module 101, a cross-modal fusion module 102, a dynamic correlation analysis module 103, a health scoring module 104, and a report output module 105. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0107] In this embodiment, the functions of each module / unit are as follows: The image feature data acquisition module 101 is used to perform weight allocation on the image feature data of building equipment to obtain weighted image feature data. The cross-modal fusion module 102 is used to fuse the weighted image feature data with the standardized parameters element by element to obtain the cross-modal representation of the building equipment. The dynamic correlation analysis module 103 is used to perform dynamic correlation analysis on the mechanical operation characteristics and electrical operation characteristics in the cross-modal representation to obtain the state coupling feature set of the building equipment. The health scoring module 104 is used to map the state coupling feature set to the comprehensive health score of the building equipment. The report output module 105 is used to output a health status report and maintenance recommendations for the building equipment based on the comprehensive health score.

[0108] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0109] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0111] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0112] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for assessing the operational status of building equipment by integrating machine vision and the Internet of Things, characterized in that, The method includes: S1. Weight the image feature data of building equipment to obtain weighted image feature data; S2. The weighted image feature data is fused with the standardized parameters element by element to obtain the cross-modal characterization of the building equipment; S3. Perform dynamic correlation analysis on the mechanical operation characteristics and electrical operation characteristics in the cross-modal characterization to obtain the state coupling characteristic set of the building equipment; S4. Map the state coupling feature set to a comprehensive health score of the building equipment; S5. Based on the comprehensive health score, output a health status report and maintenance recommendations for the building equipment.

2. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 1, characterized in that, The weighted image feature data of the building equipment is obtained by weighting the image feature data, including: Spatial feature decoupling is performed on the image feature data to obtain the key appearance feature regions of the building equipment; Defect risk analysis is performed on the key appearance feature areas to obtain the preliminary risk weight of the building equipment; Multimodal collaborative optimization is performed on the preliminary risk weights to obtain the dynamic weight coefficients of the building equipment; The image feature data is dynamically weighted and fused based on the dynamic weight coefficients to obtain weighted image feature data.

3. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 1, characterized in that, The step of fusing the weighted image feature data with the standardized parameters element by element includes: Based on the weighted image feature data, the standardized parameters are dimension-aligned to obtain a dimension-matched standardized parameter matrix; Tensor fusion is performed on the weighted image feature data and the standardized parameter matrix to obtain the primary fusion feature matrix of the building equipment; The primary fusion feature matrix is ​​subjected to cross-modal feature transformation to obtain enhanced cross-modal intermediate features.

4. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 3, characterized in that, The step of performing cross-modal feature transformation on the primary fused feature matrix to obtain enhanced cross-modal intermediate features includes: Channel-level feature fusion is performed on each channel in the cross-modal intermediate features to obtain the dynamic health status vector of the building equipment; Construct a channel attention weight matrix based on the dynamic health status vector; Based on the channel attention weight matrix, the cross-modal intermediate features are subjected to attention-weighted fusion to obtain the optimized cross-modal representation; The optimized cross-modal feature representation is subjected to layer normalization to obtain the cross-modal characterization of the building equipment.

5. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 1, characterized in that, The dynamic correlation analysis of the mechanical and electrical operating characteristics in the cross-modal characterization yields the state coupling feature set of the building equipment, including: The current fluctuation sequence in the electrical operating characteristics is time-domain aligned to obtain the aligned current fluctuation sequence. Based on the coupling coefficient formula, the dynamic correlation between the aligned current fluctuation sequence and the vibration amplitude sequence is calculated to obtain the state coupling feature set of the building equipment, wherein the coupling coefficient formula is: ， In the formula, For dynamic correlation, It is a sequence of vibration amplitudes. It is a current fluctuation sequence. This represents the instantaneous change in the vibration amplitude sequence. This represents the instantaneous change in the current fluctuation sequence. The standard deviation of the vibration amplitude sequence. denoted as the standard deviation of the current fluctuation sequence.

6. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 5, characterized in that, The step of performing time-domain alignment on the current fluctuation sequence in the electrical operating characteristics to obtain the aligned current fluctuation sequence includes: Phase offset compensation is performed on the reference time axis of the current fluctuation sequence to obtain the phase offset compensated current fluctuation sequence; The current fluctuation sequence after phase offset compensation is normalized to obtain a time-domain aligned current fluctuation sequence.

7. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 1, characterized in that, The process of mapping the state-coupled feature set to a comprehensive health score for the building equipment includes: The dynamic health indicators of the building equipment are obtained by weighted fusion of the multi-source dynamic characteristics in the state coupling feature set. The dynamic health indicators are reduced by a vital sign scaling method to obtain a multi-level health score for the building equipment. The multi-level health scores are aggregated to obtain the comprehensive health score of the building equipment.

8. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 7, characterized in that, The process of scaling down the dynamic health indicators to obtain a multi-level health score for the building equipment includes: When the dynamic health indicator is in a stable range, a high health score is output. When the dynamic health indicator is in the transition range, a medium health score is output. When the dynamic health indicator is in an abnormal range, a low health score is output.

9. The method for evaluating the operational status of building equipment integrating machine vision and the Internet of Things as described in claim 1, characterized in that, The process of outputting a health status report and maintenance recommendations for the building equipment based on the comprehensive health score includes: The comprehensive health score is matched with a preset health reference value, and the matching result is encoded into a corresponding report generation instruction. Based on the report generation instructions, a health status report and maintenance recommendations for the building equipment are generated.

10. A building equipment operation status assessment system integrating machine vision and the Internet of Things, characterized in that, The system includes: The image feature data acquisition module is used to assign weights to the image feature data of building equipment to obtain weighted image feature data. The cross-modal fusion module is used to fuse the weighted image feature data with the standardized parameters element by element to obtain the cross-modal representation of the building equipment; The dynamic correlation analysis module is used to perform dynamic correlation analysis on the mechanical operation characteristics and electrical operation characteristics in the cross-modal representation to obtain the state coupling feature set of the building equipment. A health rating module is used to map the state coupling feature set to a comprehensive health rating of the building equipment; The report output module is used to output a health status report and maintenance recommendations for the building equipment based on the comprehensive health score.

Images