Building operation and maintenance management and control method based on image recognition technology

Abstract

The invention discloses a building operation and maintenance management and control method based on an image recognition technology. The method comprises the following steps: constructing a space object model; establishing an equipment and personnel element target detection neural network model; personnel detection and equipment state judgment; formulating and implementing a management and control scheme of electromechanical equipment in the building; and updating a historical database and perfecting a management and control strategy. On-off states of personnel and electromechanical equipment in a building are identified based on an image identification technology, a management and control strategy of the electromechanical equipment is implemented according to the detected personnel change and the on-off state of the electromechanical equipment, and the management and control strategy is optimized and adjusted according to the satisfaction degree condition to improve the management and control strategy. The equipment can be ensured to operate in an optimal state so as to continuously monitor and improve the building, energy-saving management and intelligent operation and maintenance of the building are realized, and the problems of high energy consumption, energy waste caused by invalid opening, difficulty in manual inspection and monitoring, high time cost, untimely operation and low cost of old or complex-space-layout buildings are solved. And the cost of deploying sensor monitoring electromechanical equipment is high.

Description

Technical Field

[0001] The present invention relates to the field of building operation and maintenance, and in particular to a building operation and maintenance management and control method based on image recognition technology. Background Art

[0002] In the field of building operation and maintenance, Building Information Modeling (BIM) has become an important method for building intelligent operation and maintenance systems in modern large buildings. By integrating the three-dimensional geometric information of the building and the detailed information of the equipment in the corresponding area, and combining real-time data such as temperature, humidity, and light sensors to establish a BIM operation and maintenance model, a unified data platform is formed, which provides a scientific decision-making basis for the operation and maintenance of the building. However, not all large public buildings can easily and quickly establish intelligent operation and maintenance systems. In the face of some old buildings or buildings with complex spatial layouts, it is often very difficult to supervise the operation of equipment, and it is common for equipment to be ineffectively turned on, resulting in energy waste. Routine inspections take up a lot of time for maintenance personnel and are not timely, while deploying a large number of sensors will require a high cost. Summary of the invention

[0003] The purpose of the present invention is to provide a building operation and maintenance control method based on image recognition technology to solve the problems of ineffective activation of electromechanical equipment in some old buildings or buildings with complex spatial layouts, resulting in energy waste, difficulty in manual inspection and monitoring of equipment, high time cost and untimely, and high cost of deploying sensors to monitor electromechanical equipment.

[0004] In order to solve the above technical problems, the present invention provides a building operation and maintenance control method based on image recognition technology, comprising:

[0005] Step 1: Construct a spatial object model;

[0006] Step 2: Establish a neural network model for equipment and personnel element target detection;

[0007] Step 3: Personnel detection and equipment status judgment;

[0008] Step 4: Develop and implement a control plan for mechanical and electrical equipment in the building;

[0009] Step 5: Update the historical database and improve the management and control strategy.

[0010] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in the step 1, the method for constructing a spatial object model includes: establishing a BIM operation and maintenance model of a building containing electromechanical equipment on a building maintenance system server, wherein the electromechanical equipment includes lighting equipment, building maintenance information display screens and air conditioners, wherein the electromechanical equipment parameters are E qi , including the spatial parameter Si and device setting parameters C i .

[0011] Furthermore, the building operation and maintenance management method based on image recognition technology provided by the present invention is characterized in that, in the step 1, the method of constructing a BIM operation and maintenance model of a building containing electromechanical equipment includes:

[0012] Step 1.1: Based on the relevant building data, establish a BIM operation and maintenance model of the building containing mechanical and electrical equipment on the construction and maintenance system server;

[0013] Step 1.2: The equipment setting parameters are determined based on historical data, including the air conditioning setting temperature, the normal operation time period of the maintenance information display screen, the lighting equipment setting brightness and lighting time period.

[0014] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in step 2, the method of establishing a neural network model for equipment and personnel element target detection includes:

[0015] Step 2.1: The building maintenance system server uses the existing video surveillance equipment in the building to obtain image data of different time periods and different areas, and establishes a personnel factor image dataset and an equipment factor image dataset respectively;

[0016] Step 2.2: Annotate the electromechanical devices in the device feature image dataset and randomly divide them into training set, validation set and test set according to proportion;

[0017] Step 2.3: Construct a deep learning neural network model for target detection, train the deep learning neural network model based on the personnel factor image dataset and the equipment factor image dataset, and obtain a personnel factor target detection neural network model and an equipment factor target detection neural network model that meet the requirements of accuracy, recall rate and detection precision.

[0018] Furthermore, the building operation and maintenance management method based on image recognition technology provided by the present invention is characterized in that, in the step 2.1, the method of establishing the personnel element image dataset and the equipment element image dataset includes: for the personnel element image dataset, it is necessary to cover a variety of condition scenes, wherein the various scenes include different lighting conditions, personnel densities and personnel activities; for the equipment element image dataset, it is necessary to cover a variety of status scenes, wherein the various status scenes include different equipment operating states, different angles and backgrounds, wherein the equipment operating state includes on and off.

[0019] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in step 3, the method of personnel detection and equipment status judgment includes:

[0020] Step 3.1: Get a screenshot of the video stream of the image data in step 2.1, and feed the screenshot to the human factor target detection neural network model and the device factor target detection neural network model obtained in step 2 respectively;

[0021] Step 3.2: For lighting equipment and construction and maintenance information display screens in electromechanical equipment, grayscale processing is performed on the images in which the lighting equipment and construction and maintenance information display screens are identified in the equipment element target detection neural network model, and the brightness information is retained:

[0022] Step 3.2.1: Identify the color information of the image according to formula (1);

[0023] I(x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y) (1);

[0024] Where I(x,y) represents the position of image I, R(x,y), G(x,y), B(x,y) represent the red, green and blue channel values ​​of image I at position (x,y) respectively;

[0025] Step 3.3.2: The location where the lighting equipment and the maintenance information display screen are identified is selected as the region of interest, and the average brightness of the region of interest is calculated according to formula (2);

[0026]

[0027] Where L is the average brightness, ROI is the region of interest, i is the brightness value of the i-th pixel in the region of interest, and N is the total value of the pixels in the region of interest;

[0028] The building operation and maintenance server determines the switch status of the lighting equipment and the building maintenance information display screen in the electromechanical equipment according to the average brightness value of formula (2). When the average brightness value exceeds the predetermined brightness threshold, it is considered that the lighting equipment and the building maintenance information display screen are in the on state, otherwise they are in the off state;

[0029] Step 3.3: For the air conditioners in the electromechanical equipment, a cloth strip is tied at the air outlet of each air conditioner, and the position information of the cloth strip is identified through the equipment element target detection neural network model; according to the position change of the cloth strip in the continuous image recognition results of the previous and next frames, the displacement of the cloth strip is calculated to determine its motion state. The building operation and maintenance server determines the switch state of the air conditioner in the electromechanical equipment according to the displacement of the cloth strip. When the displacement of the cloth strip exceeds the displacement threshold, the air conditioner is considered to be in the on state, otherwise it is in the off state;

[0030] Step 3.4: For personnel, considering their movement characteristics, the personnel element target detection neural network model uses the optical flow method based on image recognition technology to detect moving objects and thus identify personnel.

[0031] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in the step 3.4, a filter is constructed for the detected personnel, and a kernel correlation filter algorithm tracking algorithm is used to count them.

[0032] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in step 4, the method of formulating and implementing a control plan for electromechanical equipment in a building includes:

[0033] Step 4.1: According to the spatial parameter S i Classification is performed according to the time parameter T i , personnel parameter P i , combined with the equipment setting parameter C i , construct management rules Mi = {S i ,T i ,P i ,C i};

[0034] Step 4.2: Implement control strategies:

[0035] The building maintenance system server reads the rule engine on the computer room server of the existing video surveillance equipment in the building and initiates an alarm for abnormal status of electromechanical equipment;

[0036] When the space parameter S i If no personnel are identified, determine whether all devices are in the off state according to step 3. If any device is in the on state, the maintenance system server determines that the switch state of the corresponding device is abnormal and issues an alarm message;

[0037] When the space parameter S i If a person is identified outside normal working hours, it is determined according to step 3 whether all devices are in the off state. If a device is in the on state, the construction and maintenance system server determines that the switch state of the corresponding device is abnormal and issues an alarm message;

[0038] When the space parameter S i When it is recognized that the number of people increases during normal working hours, the construction and maintenance system server sends a notification message to the building operation and maintenance management personnel that the air conditioner should be cooled down by 1°C, and records the change; when the number of people decreases, the construction and maintenance system server sends a notification message to the building operation and maintenance management personnel that the air conditioner should be heated up by 1°C, and records the change.

[0039] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in step 4.2, when the cumulative number of temperature drops exceeds the set value or the threshold set by the state, information that the temperature adjustment has reached the threshold is sent to the building operation and maintenance management personnel; when the cumulative number of temperature increases exceeds the set value or the threshold set by the state, information that the temperature adjustment has reached the threshold is sent.

[0040] Furthermore, the building operation and maintenance control method based on image recognition technology provided by the present invention is characterized in that, in step 5, the method of updating the historical database and improving the control strategy includes:

[0041] Step 5.1: Collect the satisfaction of the control strategy St according to the alarm information i , employee age i , employee gender G i , space parameter S i , time parameter T i And equipment setting parameters C i ;

[0042] Step 5.1.1: Standardize all data by category according to formula (3):

[0043]

[0044] Among them, X scaled is the data standardized value, X is the original eigenvalue (S i , T i , Age i , G i , C i ), τ is the mean of the feature, σ is the standard deviation of the feature;

[0045] Step 5.1.2: Create interaction terms to capture the mutual influence between the parameters in the original eigenvalues ​​and obtain (S i ×T i ) scaled ,(S i ×Age i ) scaled ,(S i ×G i ) scaled ,(Age i ×T i ) scaled ,(G i ×T i ) scaled ,(Age i ×G i ) scaled ,(S i ×C i) scaled ,(C i ×Age i ) scaled ,(T i ×C i ) scaled ,(G i ×C i ) scaled ;

[0046] Step 5.1.3: Create polynomial terms based on the nonlinear relationship between the original eigenvalue and satisfaction And build the feature matrix:

[0047]

[0048] Step 5.1.4: Input the K-means clustering algorithm into the feature matrix and minimize the sum of the squares of the distances from each data point to the center of its cluster according to formula (4);

[0049]

[0050] Where k is the number of clusters, x j is a data point, C i is cluster i, μ i is the center of the cluster;

[0051] According to formula (4), the factors that affect the satisfaction of the control strategy are analyzed, and different samples are clustered to identify groups with similar characteristics and behavior patterns. Based on cluster analysis, it is found how different parameters jointly affect the satisfaction of the control strategy, providing guidance for optimizing the control strategy;

[0052] Step 5.1.5: Use the silhouette coefficient to evaluate the clustering quality according to formula (5):

[0053]

[0054] Among them, a is the average distance from a data point to other points in its cluster, b is the average distance from a data point to the nearest cluster, calculate the silhouette coefficients of different clusters, and select the optimal number of clusters;

[0055] According to the value of clustering quality evaluation in formula (5), determine whether the satisfaction level is consistent with the actual situation;

[0056] Step 5.1.6: Calculate the mean value of each cluster group according to formula (6);

[0057]

[0058] Among them, Avg i is the feature mean of each cluster, n iis the number of data points in cluster i;

[0059] According to formula (6), which reflects the relationship between this parameter and satisfaction, the characteristic mean of each cluster is correlated with the strategy satisfaction Sti. Parameters with negative characteristic mean values ​​indicate that the parameter will lead to a decrease in satisfaction and need to be adjusted in the relevant management and control strategies; parameters with positive characteristic mean values ​​indicate that the parameter meets the needs of the corresponding population.

[0060] Step 5.2: Through the feature matrix of step 5.1, find the key issues that affect personnel satisfaction and improve the corresponding management and control strategies; at the same time, update the setting parameters of the equipment to ensure that the equipment can operate in the best state:

[0061] When the time parameter T i The characteristic value of is negative, indicating that the employees are not satisfied with the time setting. The following policy adjustments are performed to improve the control strategy:

[0062] Strategy A: In areas where the number of device shutdown alarms due to absence exceeds the normal threshold, the device startup time needs to be reduced;

[0063] Strategy B, in areas where shutdown alarms are issued multiple times after get off work hours, increase the operating time of the equipment;

[0064] When the device sets parameter C i The characteristic value of is negative, indicating that the employee is not satisfied with the equipment setting value. The following adjustment strategies are implemented to improve the control strategy:

[0065] Strategy C: For areas with frequent temperature drop alarms, lower the starting set temperature of the air conditioner;

[0066] When the characteristic values ​​of employee parameters, age, and gender are negative, execute the following adjustment strategies to improve the control strategy:

[0067] Strategy D: For areas where older employees gather, increase the air conditioning setting temperature;

[0068] Strategy E: For areas where female employees gather, increase the air conditioning setting temperature;

[0069] When the space parameter S i The characteristic value of is negative, and the following adjustment strategies are implemented to improve the control strategy:

[0070] Strategy F: The space in this area is large or poorly sealed, the air conditioning has poor cooling effect, and the lighting equipment has poor lighting effect. Lower the air conditioning set temperature and increase the number of lighting equipment.

[0071] Based on all relevant data, including analysis results, adjusted setting parameters and historical alarm information, an electromechanical equipment operation and maintenance information database is established on the building and maintenance system server and its data is updated regularly to ensure continuous monitoring and improvement of buildings and realize building energy-saving management and intelligent operation and maintenance.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] The building operation and maintenance control method based on image recognition technology provided by the present invention is aimed at old buildings or buildings with complex spatial layouts. By using the existing video surveillance equipment in the building, image data of different time periods and different areas are obtained, and the switch status of personnel and electromechanical equipment in the building is identified based on image recognition technology. The control strategy of electromechanical equipment is implemented according to the detected personnel changes and the switch status of electromechanical equipment. The control strategy is optimized and adjusted according to the satisfaction level to improve the control strategy, so as to ensure that the equipment can operate in the best state, so as to continuously monitor and improve the building, realize building energy-saving management and intelligent operation and maintenance, avoid the problems of energy waste caused by invalid opening, difficult and time-consuming manual inspection and monitoring of equipment, and high cost and delay, and high cost of deploying sensors to monitor electromechanical equipment. Through the existing video surveillance equipment in the building, the equipment status is obtained using image recognition technology, and then combined with the static data in the BIM operation and maintenance model, the abnormal equipment identification is realized and a historical database is formed to assist scientific decision-making, reduce operation and maintenance costs, and improve management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a flow chart of the building operation and maintenance control method based on image recognition technology. DETAILED DESCRIPTION

[0075] The present invention is described in detail below in conjunction with the accompanying drawings: The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0076] Please refer to Figure 1 The embodiment of the present invention provides a building operation and maintenance control method based on image recognition technology, comprising:

[0077] Step 1: Construct a spatial object model, including establishing a BIM operation and maintenance model of a building containing electromechanical equipment on the construction and maintenance system server. The electromechanical equipment includes lighting equipment, construction and maintenance information display screens, and air conditioners. The electromechanical equipment parameters are E qi , including the spatial parameter S i and device setting parameters C i To improve the accuracy of the BIM operation and maintenance model, the following can be included:

[0078] Step 1.1: Based on the building-related data, establish a BIM operation and maintenance model of the building containing mechanical and electrical equipment on the construction and maintenance system server.

[0079] Step 1.2: The equipment setting parameters are determined based on historical data, including the air conditioning setting temperature, the normal operation time period of the maintenance information display screen, the lighting equipment setting brightness and lighting time period.

[0080] Step 2: Establish a neural network model for equipment and personnel element target detection, including:

[0081] Step 2.1: The construction and maintenance system server uses the existing video surveillance equipment in the building to obtain image data of different time periods and different areas, and establishes the personnel element image data set and the equipment element image data set respectively. The construction and maintenance system server is connected to the computer room server of the existing video surveillance equipment in the building. For the personnel element image data set, it is necessary to cover a variety of condition scenes, including different lighting conditions, personnel density and personnel activities; for the equipment element image data set, it is necessary to cover a variety of state scenes, including different equipment operation states, different angles and backgrounds, and the equipment operation state includes the switch state of on and off.

[0082] Step 2.2: Label the electromechanical devices in the device element image dataset and randomly divide them into training set, validation set and test set in proportion. Labeling tools such as LabelImg and VGG Image Annotator can be used to label the electromechanical devices in the device element image dataset and randomly divide them into training set, validation set and test set in proportion. Among them:

[0083] The training set is used to fit the model. By setting the parameters of the classifier, the classification model is trained. When combined with the validation set, different values ​​of the same parameter will be selected to fit multiple classifiers.

[0084] The purpose of the cross validation set is to use each model to predict the validation set data and record the model accuracy in order to find the best model after multiple models are trained through the training set. The parameters corresponding to the best model are selected to adjust the model parameters, such as the parameter c and kernel function in svn.

[0085] Test set: After the optimal model is obtained through the training set and the validation set, the test set is used for model prediction. It is used to measure the performance and classification ability of the optimal model. In other words, the test set can be regarded as a data set that has never existed. When the model parameters have been determined, the test set is used to evaluate the model performance.

[0086] Step 2.3: Construct a deep learning neural network model for target detection, train the deep learning neural network model based on the personnel factor image dataset and the equipment factor image dataset, and obtain a personnel factor target detection neural network model and an equipment factor target detection neural network model that meet the requirements of accuracy, recall rate and detection precision.

[0087] Step 3: Personnel detection and equipment status judgment, including:

[0088] Step 3.1: Get a video stream screenshot of the image data in step 2.1, and feed the screenshot to the human factor target detection neural network model and the device factor target detection neural network model obtained in step 2. The video stream screenshot can be obtained by using a video screenshot tool such as FFMPEG.

[0089] Step 3.2: For lighting equipment and construction and maintenance information display screens in electromechanical equipment, grayscale the images in which the lighting equipment and construction and maintenance information display screens are identified in the equipment element target detection neural network model, and retain the brightness information.

[0090] Step 3.2.1: Identify the color information of the image according to formula (1).

[0091] I(x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y) (1).

[0092] Where I(x,y) represents the position of image I, R(x,y), G(x,y), B(x,y) represent the red, green and blue channel values ​​of image I at position (x,y), respectively.

[0093] Step 3.3.2: The location where the lighting equipment and the maintenance information display screen are identified is selected as the region of interest, and the average brightness of the region of interest is calculated according to formula (2).

[0094]

[0095] Where L is the average brightness, ROI is the region of interest, i is the brightness value of the i-th pixel in the region of interest, and N is the total value of the pixels in the region of interest.

[0096] The building operation and maintenance server determines the switch status of the lighting equipment and the building maintenance information display screen in the electromechanical equipment according to the average brightness value of formula (2). When the average brightness value exceeds the predetermined brightness threshold, it is considered that the lighting equipment and the building maintenance information display screen are in the on state, otherwise they are in the off state.

[0097] Step 3.3: For the air conditioners in the electromechanical equipment, a cloth strip is tied at the air outlet of each air conditioner, and the position information of the cloth strip is identified through the equipment element target detection neural network model; according to the position change of the cloth strip in the continuous image recognition results of the previous and next frames, the displacement of the cloth strip is calculated to determine its motion state. The building operation and maintenance server determines the switch state of the air conditioner in the electromechanical equipment according to the displacement of the cloth strip. When the displacement of the cloth strip exceeds the displacement threshold, the air conditioner is considered to be in the on state, otherwise it is in the off state. The cloth strip can be red or other colors that are easy to identify.

[0098] Step 3.4: For people, considering their motion characteristics, the moving objects are detected by the optical flow method based on the image recognition technology through the neural network model of the human element target detection, so as to identify the people. In order to improve the accuracy of identifying people and avoid misjudgment, the detected people are counted by building a filter and using the kernel correlation filter (KCF) tracking algorithm.

[0099] Step 4: Develop and implement a control plan for mechanical and electrical equipment in the building, including:

[0100] Step 4.1: According to the spatial parameter S i Classification is performed according to the time parameter T i , personnel parameter P i , combined with the equipment setting parameter C i , construct management rules Mi = {S i ,T i ,P i ,C i}.

[0101] Step 4.2: Implement control strategies.

[0102] The building maintenance system server reads the rule engine on the computer room server of the existing video surveillance equipment in the building and initiates an alarm for abnormal status of mechanical and electrical equipment.

[0103] When the space parameter S i If no personnel are identified, determine whether all devices are in the off state according to step 3. If any device is in the on state, the construction and maintenance system server determines that the switch state of the corresponding device is abnormal and issues an alarm message.

[0104] When the space parameter S i If a person is identified outside normal working hours, it is determined according to step 3 whether all devices are in the off state. If a device is in the on state, the building maintenance system server determines that the switch state of the corresponding device is abnormal and issues an alarm message. The alarm information can provide guidance for the building operation and maintenance managers on the operation and maintenance of the electromechanical equipment in the building.

[0105] When the space parameter S i When it is recognized that the number of people increases during the normal working period, the construction and maintenance system server sends a notification message to the building operation and maintenance management personnel that the air conditioner should be cooled down by 1°C, and records the change; in step 4.2, when the cumulative number of temperature reductions exceeds the set value or the threshold set by the state, a message that the temperature adjustment has reached the threshold is sent to the building operation and maintenance management personnel; when the number of people decreases, the construction and maintenance system server sends a notification message to the building operation and maintenance management personnel that the air conditioner should be heated up by 1°C, and records the change; when the cumulative number of temperature increases exceeds the set value or the threshold set by the state, a message that the temperature adjustment has reached the threshold is sent.

[0106] Step 5: Update the historical database and improve the management and control strategy, including:

[0107] Step 5.1: Collect the satisfaction of the control strategy St according to the alarm information i , employee age i , employee gender G i , space parameter S i , time parameter T i And equipment setting parameters C i .

[0108] Step 5.1.1: Standardize all data by category according to formula (3).

[0109]

[0110] Among them, X scaled is the data standardized value, X is the original eigenvalue (S i , T i , Age i , G i , C i ), τ is the mean of the feature, and σ is the standard deviation of the feature.

[0111] Step 5.1.2: Create interaction terms to capture the mutual influence between the parameters in the original eigenvalues ​​and obtain (S i ×T i ) scaled ,(S i ×Age i ) scaled ,(S i ×G i ) scaled ,(Age i ×T i ) scaled ,(G i ×T i ) scaled ,(Age i×G i ) scaled ,(S i ×C i ) scaled ,(C i ×Age i ) scaled ,(T i ×C i ) scaled ,(G i ×C i ) scaled .

[0112] Step 5.1.3: Create polynomial terms based on the nonlinear relationship between the original eigenvalue and satisfaction And build the feature matrix:

[0113]

[0114] Each row in the feature matrix corresponds to a data point, and each column corresponds to a feature.

[0115] Step 5.1.4: Input the K-means clustering algorithm into the feature matrix and minimize the sum of the squared distances from each data point to the center of its cluster according to formula (4).

[0116]

[0117] Where k is the number of clusters, x j is a data point, C i is cluster i, μ i is the center of the cluster.

[0118] According to formula (4), the factors that affect the satisfaction with the management and control strategy are analyzed, and different samples are clustered to identify groups with similar characteristics and behavior patterns. Based on cluster analysis, it is found how different parameters jointly affect the satisfaction with the management and control strategy, providing guidance for optimizing the management and control strategy.

[0119] Step 5.1.5: Use the Silhouette Score to evaluate the clustering quality according to formula (5).

[0120]

[0121] Among them, a is the average distance from a data point to other points in its cluster, b is the average distance from a data point to the nearest cluster, calculate the silhouette coefficients of different clusters, and select the optimal number of clusters.

[0122] According to the clustering quality evaluation value of formula (5), it is judged whether the satisfaction level is consistent with the actual situation.

[0123] Formula (5) uses a clustering algorithm to infer the relationship between variables and satisfaction. It is a sense of distribution around the correct result and has accuracy. For the inferred relationship, the score is calculated using formula (5). The higher the score, the more accurate the inference is and the more consistent it is with the actual situation of the impact of these parameters on satisfaction.

[0124] Step 5.1.6: Calculate the mean value of each cluster group according to formula (6).

[0125]

[0126] Among them, Avg i is the feature mean of each cluster, n i is the number of data points in cluster i;

[0127] The characteristic mean represents the center point of the cluster, which can represent the overall characteristics of the cluster. According to formula (6), the relationship between this parameter and satisfaction is reflected. The characteristic mean of each cluster is correlated with the strategy satisfaction Sti. The parameter with a negative characteristic mean indicates that the parameter will lead to a decrease in satisfaction and needs to be adjusted in the relevant management and control strategy; the parameter with a positive characteristic mean indicates that the parameter meets the needs of the corresponding population.

[0128] Step 5.2: Use the feature matrix of step 5.1 to find the key issues that affect employee satisfaction and improve the corresponding management and control strategies; at the same time, update the equipment setting parameters to ensure that the equipment can operate in the best condition.

[0129] When the time parameter T i The characteristic value of is negative, indicating that the employees are not satisfied with the time setting. The following policy adjustments are performed to improve the control strategy:

[0130] Strategy A: In areas where the number of device shutdown alarms due to absence exceeds the normal threshold, the device startup time needs to be reduced.

[0131] Strategy B, in areas where shutdown alarms are issued multiple times outside of working hours, increase the operating time of the equipment.

[0132] When the device sets parameter C i The characteristic value of is negative, indicating that the employee is not satisfied with the equipment setting value. The following adjustment strategies are implemented to improve the control strategy:

[0133] Strategy C: In areas where temperature drop alarms are frequently issued, the starting set temperature of the air conditioner is lowered.

[0134] When the characteristic values ​​of employee parameters, age, and gender are negative, execute the following adjustment strategies to improve the control strategy:

[0135] Strategy D: Increase the air conditioning setting temperature in areas where older employees gather.

[0136] Strategy E: For areas where female employees gather, increase the air conditioning setting temperature.

[0137] When the space parameter S i The characteristic value of is negative, and the following adjustment strategies are implemented to improve the control strategy:

[0138] Strategy F: The space in this area is large or poorly sealed, the air conditioning has poor cooling effect, and the lighting equipment has poor lighting effect. Lower the air conditioning set temperature and increase the number of lighting equipment.

[0139] Based on all relevant data, including analysis results, adjusted setting parameters and historical alarm information, an electromechanical equipment operation and maintenance information database is established on the building and maintenance system server and its data is updated regularly to ensure continuous monitoring and improvement of buildings and realize building energy-saving management and intelligent operation and maintenance.

[0140] The building operation and maintenance control method based on image recognition technology provided by the embodiment of the present invention is aimed at old buildings or buildings with complex spatial layouts. By using the existing video surveillance equipment in the building, image data of different time periods and different areas are obtained, and the personnel and the switch status of electromechanical equipment in the building are identified based on image recognition technology. The control strategy of electromechanical equipment is implemented according to the detected personnel changes and the switch status of electromechanical equipment. The control strategy is optimized and adjusted according to the satisfaction level to improve the control strategy, so as to ensure that the equipment can operate in the best state, so as to continuously monitor and improve the building, realize building energy-saving management and intelligent operation and maintenance, and avoid the problems of energy waste caused by invalid opening, the difficulty and high time cost of manual inspection and monitoring of equipment, and the high cost of deploying sensors to monitor electromechanical equipment. Through the existing video surveillance equipment in the building, the equipment status is obtained using image recognition technology, and then combined with the static data in the BIM operation and maintenance model, the abnormal equipment identification is realized and a historical database is formed to assist scientific decision-making, reduce operation and maintenance costs, and improve management efficiency.

[0141] The building operation and maintenance control method based on image recognition technology provided by the embodiment of the present invention can improve the energy-saving effect of the building: by actively identifying abnormal scenes, targeted and timely processing, and reducing energy waste. It can also improve the work efficiency of maintenance personnel: reduce ineffective inspection work. It can also improve the management level: by summarizing the collected status data, scientific decision-making can be made.

[0142] The present invention is not limited to the above-mentioned specific implementation modes. Obviously, the above-mentioned embodiments are only some embodiments of the embodiments of the present invention, but not all embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention. Those skilled in the art can make other levels of modifications and changes to the present invention. In this way, if these modifications and changes of the present invention fall within the scope of the claims of the present invention, the present invention is also intended to include these changes and changes.

Claims

1. A building operation and maintenance control method based on image recognition technology, characterized in that: include: Step 1: Build a spatial object model; Step 2: Establish a neural network model for equipment and personnel element target detection; Step 3: Personnel detection and equipment status judgment; Step 4: Develop and implement a control plan for mechanical and electrical equipment in the building; Step 5: Update the historical database and improve the management and control strategy.

2. The building operation and maintenance control method based on image recognition technology according to claim 1 is characterized in that: In step 1, the method for constructing a spatial object model includes: establishing a BIM operation and maintenance model of a building containing electromechanical equipment on a construction and maintenance system server, wherein the electromechanical equipment includes lighting equipment, construction and maintenance information display screens, and air conditioners, wherein the electromechanical equipment parameters are E qi , including the spatial parameter S i and device setting parameters C i .

3. The building operation and maintenance control method based on image recognition technology according to claim 2 is characterized in that: In step 1, the method for constructing a BIM operation and maintenance model of a building containing electromechanical equipment includes: Step 1.1: Based on the relevant building data, establish a BIM operation and maintenance model of the building containing mechanical and electrical equipment on the construction and maintenance system server; Step 1.2: The equipment setting parameters are determined based on historical data, including the air conditioning setting temperature, the normal operation time period of the maintenance information display screen, the lighting equipment setting brightness and lighting time period.

4. The building operation and maintenance control method based on image recognition technology according to claim 2 is characterized in that: In step 2, the method for establishing a neural network model for detecting equipment and personnel elements includes: Step 2.1: The building maintenance system server uses the existing video surveillance equipment in the building to obtain image data of different time periods and different areas, and establishes a personnel factor image dataset and an equipment factor image dataset respectively; Step 2.2: Annotate the electromechanical devices in the device feature image dataset and randomly divide them into training set, validation set and test set according to proportion; Step 2.3: Construct a deep learning neural network model for target detection, train the deep learning neural network model based on the personnel factor image dataset and the equipment factor image dataset, and obtain a personnel factor target detection neural network model and an equipment factor target detection neural network model that meet the requirements of accuracy, recall rate and detection precision.

5. The building operation and maintenance control method based on image recognition technology according to claim 4 is characterized in that: In step 2.1, the method for establishing a personnel element image dataset and a device element image dataset includes: for the personnel element image dataset, it is necessary to cover a variety of condition scenes, wherein the multiple scenes include different lighting conditions, personnel densities and personnel activities; for the device element image dataset, it is necessary to cover a variety of state scenes, wherein the multiple state scenes include different device operating states, different angles and backgrounds, wherein the device operating state includes on and off.

6. The building operation and maintenance control method based on image recognition technology according to claim 4 is characterized in that: In step 3, the method of personnel detection and equipment status judgment includes: Step 3.1: Get a screenshot of the video stream of the image data in step 2.1, and feed the screenshot to the human factor target detection neural network model and the device factor target detection neural network model obtained in step 2 respectively; Step 3.2: For lighting equipment and construction and maintenance information display screens in electromechanical equipment, grayscale processing is performed on the images in which the lighting equipment and construction and maintenance information display screens are identified in the equipment element target detection neural network model, and the brightness information is retained: Step 3.2.1: Identify the color information of the image according to formula (1); I(x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y)(1); Where I(x,y) represents the position of image I, R(x,y), G(x,y), B(x,y) represent the red, green and blue channel values ​​of image I at position (x,y) respectively; Step 3.3.2: The location where the lighting equipment and the maintenance information display screen are identified is selected as the region of interest, and the average brightness of the region of interest is calculated according to formula (2); Where L is the average brightness, ROI is the region of interest, i is the brightness value of the i-th pixel in the region of interest, and N is the total value of the pixels in the region of interest; The building operation and maintenance server determines the switch status of the lighting equipment and the building maintenance information display screen in the electromechanical equipment according to the average brightness value of formula (2). When the average brightness value exceeds the predetermined brightness threshold, it is considered that the lighting equipment and the building maintenance information display screen are in the on state, otherwise they are in the off state; Step 3.3: For the air conditioners in the electromechanical equipment, a cloth strip is tied at the air outlet of each air conditioner, and the position information of the cloth strip is identified through the equipment element target detection neural network model; according to the position change of the cloth strip in the continuous image recognition results of the previous and next frames, the displacement of the cloth strip is calculated to determine its motion state. The building operation and maintenance server determines the switch state of the air conditioner in the electromechanical equipment according to the displacement of the cloth strip. When the displacement of the cloth strip exceeds the displacement threshold, the air conditioner is considered to be in the on state, otherwise it is in the off state; Step 3.4: For personnel, considering their movement characteristics, the personnel element target detection neural network model uses the optical flow method based on image recognition technology to detect moving objects and thus identify personnel.

7. The building operation and maintenance control method based on image recognition technology according to claim 6 is characterized in that: In step 3.4, the detected persons are counted by constructing a filter and using a kernel correlation filter algorithm tracking algorithm.

8. The building operation and maintenance control method based on image recognition technology according to claim 6 is characterized in that: In step 4, the method for formulating and implementing a control plan for electromechanical equipment in a building includes: Step 4.1: According to the spatial parameter S i Classification is performed according to the time parameter T i , personnel parameter P i , combined with the equipment setting parameter C i , construct management rules Mi = {S i ,T i ,P i ,C i }; Step 4.2: Implement control strategies: The building maintenance system server reads the rule engine on the computer room server of the existing video surveillance equipment in the building and initiates an alarm for abnormal status of electromechanical equipment; When the space parameter S i If no personnel are identified, determine whether all devices are in the off state according to step 3. If any device is in the on state, the maintenance system server determines that the switch state of the corresponding device is abnormal and issues an alarm message; When the space parameter S i If a person is identified outside normal working hours, it is determined according to step 3 whether all devices are in the off state. If a device is in the on state, the construction and maintenance system server determines that the switch state of the corresponding device is abnormal and issues an alarm message; When the space parameter S i When it is recognized that the number of people increases during normal working hours, the construction and maintenance system server sends a notification message to the building operation and maintenance management personnel that the air conditioner should be cooled down by 1°C, and records the change; when the number of people decreases, the construction and maintenance system server sends a notification message to the building operation and maintenance management personnel that the air conditioner should be heated up by 1°C, and records the change.

9. The building operation and maintenance control method based on image recognition technology according to claim 8 is characterized in that: In step 4.2, when the cumulative number of temperature drops exceeds the set value or the threshold set by the state, a message that the temperature adjustment has reached the threshold is sent to the building operation and maintenance management personnel; when the cumulative number of temperature increases exceeds the set value or the threshold set by the state, a message that the temperature adjustment has reached the threshold is sent.

10. The building operation and maintenance control method based on image recognition technology according to claim 8 is characterized in that: In step 5, the method of updating the historical database and improving the control strategy includes: Step 5.1: Collect the satisfaction of the control strategy St according to the alarm information i , employee age i , employee gender G i , space parameter S i , time parameter T i And equipment setting parameters C i ; Step 5.1.1: Standardize all data by category according to formula (3): Among them, X scaled is the data standardized value, X is the original eigenvalue (S i , T i , Age i , G i , C i ), τ is the mean of the feature, σ is the standard deviation of the feature; Step 5.1.2: Create interaction terms to capture the mutual influence between the parameters in the original eigenvalues ​​and obtain (S i ×T i ) scaled ,(S i ×Age i ) scaled ,(S i ×G i ) scaled ,(Age i ×T i ) scaled ,(G i ×T i ) scaled ,(Age i ×G i ) scaled ,(S i ×C i ) scaled ,(C i ×Age i ) scaled ,(T i ×C i ) scaled ,(G i ×C i ) scaled ; Step 5.1.3: Create polynomial terms based on the nonlinear relationship between the original eigenvalue and satisfaction And build the feature matrix: Step 5.1.4: Input the K-means clustering algorithm into the feature matrix and minimize the sum of the squares of the distances from each data point to the center of its cluster according to formula (4); Where k is the number of clusters, x j is a data point, C i is cluster i, μ i is the center of the cluster; According to formula (4), the factors that affect the satisfaction of the control strategy are analyzed, and different samples are clustered to identify groups with similar characteristics and behavior patterns. Based on cluster analysis, it is found how different parameters jointly affect the satisfaction of the control strategy, providing guidance for optimizing the control strategy; Step 5.1.5: Use the silhouette coefficient to evaluate the clustering quality according to formula (5): Among them, a is the average distance from a data point to other points in its cluster, b is the average distance from a data point to the nearest cluster, calculate the silhouette coefficients of different clusters, and select the optimal number of clusters; According to the value of clustering quality evaluation in formula (5), determine whether the satisfaction level is consistent with the actual situation; Step 5.1.6: Calculate the mean value of each cluster group according to formula (6); Among them, Avg i is the feature mean of each cluster, n i is the number of data points in cluster i; According to formula (6), which reflects the relationship between this parameter and satisfaction, the characteristic mean of each cluster is correlated with the strategy satisfaction Sti. Parameters with negative characteristic mean values ​​indicate that the parameter will lead to a decrease in satisfaction and need to be adjusted in the relevant management and control strategies; parameters with positive characteristic mean values ​​indicate that the parameter meets the needs of the corresponding population. Step 5.2: Through the feature matrix of step 5.1, find the key issues that affect personnel satisfaction and improve the corresponding management and control strategies; at the same time, update the setting parameters of the equipment to ensure that the equipment can operate in the best condition: When the time parameter T i The characteristic value of is negative, indicating that the employees are not satisfied with the time setting. The following policy adjustments are performed to improve the control strategy: Strategy A: In areas where the number of device shutdown alarms due to absence exceeds the normal threshold, the device startup time needs to be reduced; Strategy B, in areas where shutdown alarms are issued multiple times after get off work hours, increase the operating time of the equipment; When the device sets parameter C i The characteristic value of is negative, indicating that the employee is not satisfied with the equipment setting value. The following adjustment strategies are implemented to improve the control strategy: Strategy C: For areas with frequent temperature drop alarms, lower the starting set temperature of the air conditioner; When the characteristic values ​​of employee parameters, age, and gender are negative, execute the following adjustment strategies to improve the control strategy: Strategy D: For areas where older employees gather, increase the air conditioning setting temperature; Strategy E: For areas where female employees gather, increase the air conditioning setting temperature; When the space parameter S i The characteristic value of is negative, and the following adjustment strategies are implemented to improve the control strategy: Strategy F: The space in this area is large or poorly sealed, the air conditioning has poor cooling effect, and the lighting equipment has poor lighting effect. Lower the air conditioning set temperature and increase the number of lighting equipment. Based on all relevant data, including analysis results, adjusted setting parameters and historical alarm information, an electromechanical equipment operation and maintenance information database is established on the building and maintenance system server and its data is updated regularly to ensure continuous monitoring and improvement of buildings and realize building energy-saving management and intelligent operation and maintenance.

Images