Intelligent environment monitoring system based on image analysis

Abstract

The invention discloses an intelligent environment monitoring system based on image analysis, and belongs to the technical field of intelligent environment monitoring systems. Comprising the steps of collecting dormitory environment data, collecting the sanitation condition and light-out state data of a dormitory, performing preprocessing such as transcoding and cutting on the dormitory environment data, establishing a dormitory environment model by utilizing the dormitory environment data, evaluating the trained model by utilizing a part of reserved dormitory environment data, and performing adjustment according to a result. A trained model which is qualified in evaluation is deployed in actual monitoring, further movement approaching is carried out according to a feedback result, through establishment of a dormitory environment model, in the monitoring process, uncertain factors can be predicted according to a monitoring result, so that the position of a camera is adjusted again for determination, the monitoring effect is improved, and the user experience is improved. And meanwhile, repeated monitoring of the determined area can be reduced, unnecessary movement is reduced, and the monitoring efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of environmental intelligent monitoring systems, and more specifically, to an environmental intelligent monitoring system based on image analysis. Background Art

[0002] With the continuous improvement of people's living standards and consumption capabilities, the demand for smart homes is becoming increasingly strong. There is an increasingly urgent hope to be able to remotely control home appliances intelligently. Therefore, "smart home" has become extremely popular. However, after several years, smart homes have mainly focused on the intelligence and remote control of electrical appliances, and there has been insufficient research and development in the application field of the home environment. The design of this system is mainly carried out for the special application environment of college student dormitories. Because the population density in college dormitories is large and the current management mode is generally relatively old, it is difficult to effectively manage and maintain the environmental hygiene, work and rest rules, and public order in student dormitories. In order to enable the dirty and messy dormitory environment to be timely supervised and strengthened, and to avoid the spread of diseases and the vicious cycle of bad habits caused by the environmental hygiene of the dormitory, this design aims to develop technical applications related to smart homes, aiming at how to effectively monitor the environment of student dormitories, conduct effective intelligent analysis on it, and finally provide a set of solutions for the student management department, and intelligently provide a reference plan for optimizing the environmental management of student dormitories, and ultimately realize a new remote multi-faceted, user-friendly and relatively intelligent dormitory management plan.

[0003] The prior art document with the publication number CN105005852B provides an intelligent monitoring system for dormitory environment based on image analysis. The device includes an image acquisition module, a wireless data transmission module, a PC-side image analysis module, and a database storage module; the original dormitory environment images are collected through a camera, and the collected picture records are transmitted to the computer side through wireless data transmission. Then, through means of feature value extraction, image segmentation, and comparison, weighted detection is performed on different regions of the image. The environmental conditions inside the dormitory are evaluated by calculating the similarity of the images. Finally, corresponding prompts and optimization plans are automatically generated, and these data are saved to the database storage module for easy access and query through the Internet.

[0004] The acquisition of the dormitory environment images is achieved through an OV7670 camera. The wireless data transmission module is implemented through a WIFI module and auxiliary circuits. The means of feature value extraction, image segmentation, and comparison are as follows: first, feature extraction is realized through an image histogram, then image segmentation of the image region is realized through a region splitting and merging algorithm, and then comparison of the segmented regions is performed through a histogram matching method. The comparison of the segmented regions specifically realizes the similarity of the images by calculating the histogram mean and standard deviation, and finally weighted detection of different regions is realized based on the comparison results.

[0005] Although the existing technical solutions mentioned above can achieve beneficial effects related to the existing technology through the structure of the existing technology, there are still the following defects: The device monitors through a fixed camera. During use, due to different positions, there are monitoring blind spots, the monitoring effect is low, and at the same time, the camera is always exposed, which is not conducive to the privacy of students.

[0006] In view of the above-mentioned related technologies, the inventor believes that during the monitoring process, the position of the camera can be adjusted again according to the monitoring results to determine uncertain factors, improve the monitoring effect, and at the same time reduce unnecessary movements.

[0007] In view of this, we propose an intelligent environmental monitoring system based on image analysis. Summary of the Invention

[0008] 1. Technical problems to be solved

[0009] The purpose of this application is to provide an intelligent environmental monitoring system based on image analysis, which solves the technical problems that the device in the above background technology monitors through a fixed camera, and during use, due to different positions, there are monitoring blind spots, the monitoring effect is low, and at the same time, the camera is always exposed, which is not conducive to the privacy of students, and realizes the technical effect.

[0010] 2. Technical solutions

[0011] The technical solution of this application provides an intelligent environmental monitoring system based on image analysis, including collecting dormitory environmental data, collecting the hygiene situation and lights-out status data of the dormitory;

[0012] Preprocessing of dormitory environmental data, performing preprocessing such as transcoding and cropping on the dormitory environmental data;

[0013] Establishing dormitory environmental model training, using dormitory environmental data to establish a dormitory environmental model;

[0014] Evaluating and adjusting the dormitory environmental model, using a part of the reserved dormitory environmental data to evaluate the trained model and making adjustments according to the results;

[0015] Deploying and applying the dormitory environmental model, deploying the trained and qualified model to actual monitoring;

[0016] Execution device, making a further approach by moving according to the feedback result.

[0017] By adopting the above technical solution, through the establishment of the dormitory environment model, during the monitoring process, the uncertain factors can be predicted based on the monitoring results, and then the position of the camera can be adjusted again to determine, improving the monitoring effect. At the same time, the repeated monitoring of the determined area can be reduced, unnecessary movement can be reduced, and the monitoring efficiency can be improved.

[0018] As an alternative solution of the technical solution of this application document, the dormitory environment data is collected, and the camera is used to obtain image data, including the hygiene and tidying situation of each position in the dormitory, and the status data of each bed after the lights are turned off and the lights-off time are obtained.

[0019] As an alternative solution of the technical solution of this application document, for the preprocessing of the dormitory environment data, the dormitory environment data is labeled to distinguish different hygiene states and the states after the lights are turned off. For multi-classification problems, one-hot encoding is used to map each category to a vector, where only one element is 1 and the rest are 0;

[0020] Detect and process the missing or damaged data that may exist in the image. When the value of a pixel is missing, linear interpolation is used to estimate its value using the values of its adjacent pixels. Assume that in the horizontal direction, the left and right adjacent pixel values of the missing pixel are (P left ) and (P right ), then the value of this pixel can be calculated by linear interpolation: [Missing pixel value = P left +(Distance ratio (P right -P left ))] The distance ratio refers to the ratio of the distance between the missing pixel and the left pixel to the distance between the left and right pixels.

[0021] As an alternative solution of the technical solution of this application document, for the establishment of the dormitory environment model training, a convolutional neural network architecture is selected and trained to accurately identify the dormitory hygiene situation or the lights-off state;

[0022] The dormitory environment data is divided into a training set, which is a data set for training the dormitory environment model. The dormitory environment model learns features and patterns through the training set.

[0023] As an alternative solution of the technical solution of this application document, for the evaluation and adjustment of the dormitory environment model, the evaluation index is used for accuracy. Among the samples predicted as positive classes, the proportion of the samples that are truly positive classes. Through the formula, [Accuracy = True positives / (True positives + False positives)];

[0024] The recall rate model is the proportion of the samples that are successfully predicted as positive classes among the samples that are actually positive classes. Through the formula, [Recall rate = True positives / (True positives + False negatives)];

[0025] The harmonic mean of precision and recall, which is used to consider both factors comprehensively. Through the formula, [F1 score = 2 * precision / (precision + recall)].

[0026] As an alternative solution to the technical solution of this application document, the execution device includes

[0027] A fixing frame, which is fixedly arranged in the middle of a multi-person dormitory;

[0028] A camera, which is arranged outside the fixing frame through an extension component and moves left and right through a moving component.

[0029] 3. Beneficial effects

[0030] One or more technical solutions provided in the technical solution of this application have at least the following technical effects or advantages:

[0031] 1. By establishing a dormitory environment model in this application, during the monitoring process, uncertain factors can be predicted based on the monitoring results, and then the position of the camera can be adjusted again to be determined, improving the monitoring effect. At the same time, repeated monitoring of the determined areas can be reduced, unnecessary movements can be reduced, and the monitoring efficiency can be improved.

[0032] 2. In this application, the extension component can drive the camera to extend out of the fixing frame and retract the camera through a shutter during non-monitoring periods. This setting can not only ensure the privacy period of learning, reduce the psychological burden, but also change the position and angle at different distances by the extended distance, increasing the monitoring perspective and improving the efficiency.

[0033] 3. In this application, the moving component can move the camera to different positions, thereby monitoring different positions, increasing the monitoring coverage and accuracy. Description of the drawings

[0034] Figure 1 It is a schematic structural diagram of an environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of this application;

[0035] Figure 2 It is a schematic overall structure diagram of an environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of this application;

[0036] Figure 3 It is a bottom view of the overall structure of an environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of this application;

[0037] Figure 4 It is a sectional view of the overall structure of an environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of this application;

[0038] Figure 5 The split structure diagram of the extension component of the environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of the present application;

[0039] Figure 6 The enlarged structure diagram at position A of the environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of the present application;

[0040] Figure 7 The enlarged structure diagram at position B of the environment intelligent monitoring system based on image analysis disclosed in a preferred embodiment of the present application;

[0041] Explanation of the reference numerals in the figure: 1, fixed frame; 11, shutter; 12, pulling cavity; 13, spring; 2, multi-person dormitory; 3, camera; 31, first telescopic block; 32, first lead screw; 33, first threaded hole; 34, face gear; 35, rotating gear; 36, machine cavity; 37, first motor; 301, second telescopic block; 302, second lead screw; 303, second threaded hole; 304, mechanical cavity; 305, driving gear; 306, chain; 3001, moving groove; 3002, screw; 3003, moving block; 3004, chassis; 3005, second motor. Detailed implementation manners

[0042] The present application will be further described in detail below with reference to the accompanying drawings of the specification.

[0043] Referring to Figure 1 , the embodiment of the present application provides an environment intelligent monitoring system based on image analysis, including

[0044] Collecting dormitory environment data, collecting the hygiene situation and the data of the lights-off state of the dormitory;

[0045] Preprocessing the dormitory environment data, performing preprocessing such as transcoding and cropping on the dormitory environment data;

[0046] Establishing the training of the dormitory environment model, using the dormitory environment data to establish a dormitory environment model;

[0047] Evaluating and adjusting the dormitory environment model, using a part of the reserved dormitory environment data to evaluate the trained model and making adjustments according to the results;

[0048] Deploying and applying the dormitory environment model, deploying the trained and qualified model to the actual monitoring;

[0049] Execution device, performing further movement closer according to the feedback result.

[0050] By establishing a dormitory environment model, during the monitoring process, uncertain factors can be predicted based on the monitoring results, and then the position of the camera can be adjusted again to determine, improving the monitoring effect. At the same time, repeated monitoring of the determined area can be reduced, unnecessary movement can be reduced, and the monitoring efficiency can be improved.

[0051] Referring to Figure 1 and Figure 2 , an embodiment of the present application provides an environment intelligent monitoring system based on image analysis, which collects dormitory environment data, uses a camera to obtain image data, including the hygiene and tidying situation of various positions in the dormitory, and obtains the state data of each bed after the lights are turned off and the lights-off time.

[0052] Preprocessing of dormitory environment data: Label the dormitory environment data to distinguish different hygiene states and states after the lights are turned off. For multi-classification problems, use one-hot encoding. Map each category to a vector, where only one element is 1 and the rest are 0.

[0053] Detect and process missing or damaged data that may exist in the image. When the value of a pixel is missing, linear interpolation is used to estimate it using the values of its adjacent pixels. Assume that in the horizontal direction, the left and right adjacent pixel values of the missing pixel are (P left ) and (P right ), then the value of this pixel can be calculated by linear interpolation: [Missing pixel value = P left +(Distance ratio × (P right - P left ))]. The distance ratio refers to the ratio of the distance between the missing pixel and the left pixel to the distance between the left and right pixels.

[0054] Establish dormitory environment model training: Select the convolutional neural network (CNN) framework and train it to accurately identify the dormitory hygiene situation or the lights-off state.

[0055] Divide the dormitory environment data into a training set, which is a data set used to train the dormitory environment model. The dormitory environment model learns features and patterns through the training set.

[0056] Evaluate and adjust the dormitory environment model: Use evaluation metrics such as precision. Among the samples predicted as positive classes, the proportion of samples that are truly positive classes. Through the formula, [Precision = True positives / (True positives + False positives)];

[0057] Recall: Among the samples that are actually positive classes, the proportion of samples successfully predicted as positive classes. Through the formula, [Recall = True positives / (True positives + False negatives)];

[0058] The harmonic mean of precision and recall, which is used to comprehensively consider both, is calculated by the formula: [F1 score = 2 * precision / (precision + recall)].

[0059] The execution device includes

[0060] A fixing frame 1, which is fixedly arranged in the middle of the multi-person dormitory 2;

[0061] A camera 3, which is arranged outside the fixing frame 1 through an extension component and can move left and right through a moving component.

[0062] Refer to Figure 2 and Figure 3 , an environment intelligent monitoring system based on image analysis provided by an embodiment of the present application includes

[0063] A fixing frame 1, which is fixedly arranged in the middle of the multi-person dormitory 2;

[0064] A camera 3, which is arranged outside the fixing frame 1 through an extension component and can move left and right through a moving component.

[0065] Refer to Figure 3 , in an embodiment of the present application, an environment intelligent monitoring system based on image analysis is provided, and the diameter of the camera 3 is smaller than the width of the fixing frame 1;

[0066] A door 11 is hinged to the outside of the fixing frame 1, and the door 11 is arranged in a double-leaf manner;

[0067] A pulling cavity 12 is opened on the outside of the fixing frame 1, a spring 13 is fixedly arranged inside the pulling cavity 12, and the outer end of the spring 13 is fixedly connected to the inner wall of the door 11;

[0068] The spring 13 is arranged in an inclined structure.

[0069] Through the setting of the door 11, when it extends, it can automatically push open the door 11, and after completion, the door 11 can automatically close, improving the degree of automation. At the same time, the door 11 can block the camera to protect privacy during non-monitoring periods.

[0070] Refer to Figure 5 and Figure 6 , an environment intelligent monitoring system based on image analysis provided by an embodiment of the present application, the extension component includes a first telescopic block 31 slidably arranged inside the fixing frame 1, a first lead screw 32 is rotatably arranged inside the fixing frame 1, and the first lead screw 32 is threadedly connected to a first threaded hole 33 opened on the outer wall of the first telescopic block 31;

[0071] One end of the first lead screw 32 is fixedly provided with a face gear 34. The outer wall of the face gear 34 is meshed and connected with a rotating gear 35. A machine cavity 36 is opened inside the fixed frame 1, and a first motor 37 is arranged inside the machine cavity 36. The output shaft of the first motor 37 is coaxially and fixedly connected with the rotating gear 35.

[0072] The rotation of the output shaft of the first motor 37 drives the rotation of the rotating gear 35, and the rotating gear 35 is meshed and connected with the face gear 34, so as to make the first lead screw 32 rotate. At this time, the first telescopic block 31 threadedly connected with it performs a telescopic movement. The telescopic assembly can drive the camera to extend out of the fixed frame and retract the camera through the shutter during non-monitoring periods. This setting can not only ensure the privacy period of learning and reduce the psychological burden, but also change the angles at different distances and positions by the extended distance, increasing the monitoring perspective and improving the efficiency.

[0073] Refer to Figure 5 and Figure 7 This application embodiment provides an environment intelligent monitoring system based on image analysis. The telescopic assembly further includes a second telescopic block 301 slidably arranged inside the first telescopic block 31. A second lead screw 302 is rotatably arranged inside the first telescopic block 31. The second lead screw 302 is threadedly connected with a second threaded hole 303 opened on one side of the second telescopic block 301.

[0074] A mechanical cavity 304 is further opened inside the first telescopic block 31. A chain 306 is arranged inside the mechanical cavity 304. Driving gears 305 are meshed on both sides of the chain 306. One of the driving gears 305 is coaxially and fixedly connected with the second lead screw 302, and the other driving gear 305 is coaxially and slidably connected with the first lead screw 32.

[0075] When the first lead screw 32 rotates, it drives the coaxial driving gear 305 to rotate. The driving gear 305 is in transmission with the chain 306, so as to drive the second lead screw 302 to rotate. At this time, it drives the second telescopic block 301 threadedly connected with the second lead screw 302 to move, so as to perform secondary telescoping.

[0076] Refer to Figure 4 and Figure 5 This application embodiment provides an environment intelligent monitoring system based on image analysis. The moving assembly includes a moving groove 3001 opened on the outer wall of the second telescopic block 301. A screw 3002 is rotatably arranged inside the moving groove 3001. The screw 3002 is connected with the camera 3 through a moving block 3003 connected by threads.

[0077] A chassis 3004 is fixedly arranged inside the moving groove 3001. A second motor 3005 is arranged inside the chassis 3004. The output shaft of the second motor 3005 is coaxially and fixedly connected with the screw 3002.

[0078] The rotation of the output shaft of the second motor 3005 drives the rotation of the screw 3002. Since the screw 3002 is threadedly connected to the moving block 3003, the camera 3 is driven to move its position. The moving component enables the camera to move its position, thereby monitoring different positions and increasing the coverage and accuracy of monitoring.

[0079] When the environment intelligent monitoring system based on image analysis is required, first, collect the data on the hygiene situation and the lights-out status in the dormitory, including the hygiene and tidying-up situations at various positions in the dormitory, obtain the data on the lights-out time and the status of each bed after lights-out, perform preprocessing such as transcoding and cropping on the dormitory environment data, label the dormitory environment data to distinguish different hygiene statuses and the statuses after lights-out. For multi-classification problems, use one-hot encoding. Map each category to a vector, where only one element is 1 and the rest are 0. Detect and process the missing or damaged data that may exist in the image. When the value of a pixel is missing, linearly interpolate and estimate it using the values of its adjacent pixels. Assume that in the horizontal direction, the values of the left and right adjacent pixels of the missing pixel are (P left ) and (P right ), then the value of this pixel can be calculated by linear interpolation: [Missing pixel value = P left +(Distance ratio (P right -P left))] The distance ratio refers to the ratio of the distance between the missing pixel and the left pixel to the distance between the left and right pixels. The dormitory environment data is used to establish a dormitory environment model. The convolutional neural network (CNN) architecture is selected and trained to enable it to accurately identify the dormitory hygiene conditions or light-off status. The dormitory environment data is divided into a training set, a data set for training the dormitory environment model, and the dormitory environment model learns features and patterns through the training set. A portion of the reserved dormitory environment data is used to evaluate the trained model and adjust it according to the results; the dormitory environment model is evaluated and adjusted using the evaluation indicators. Precision: the proportion of samples predicted as positive to samples that are actually positive. The formula is [Precision = True Positives + False Positives]. Recall: the proportion of samples successfully predicted as positive to samples that are actually positive. The formula is [Recall = True Positives / (True Positives + False Negatives)]. The harmonic mean of precision and recall is used to comprehensively consider the two. The formula is [F1 score = 2Precision / (Recall Precision + Recall)]. The trained and qualified model is deployed to actual monitoring. , further move closer according to the feedback result. At this time, the rotation of the output shaft of the first motor 37 drives the rotating gear 35 to rotate, and the rotating gear 35 is meshed and connected with the face gear 34, so that the first screw rod 32 rotates. At this time, the first telescopic block 31 threadedly connected thereto performs telescopic movement. When the first screw rod 32 rotates, it drives the coaxial driving gear 305 to rotate, and the driving gear 305 and the chain 306 are transmitted, thereby driving the second screw rod 302 to rotate. At this time, the second telescopic block 301 threadedly connected to the second screw rod 302 is driven to move its position, so The secondary extension can drive the camera to extend out of the fixed frame and put away the camera through the barrier door during non-monitoring periods. This setting can not only ensure the privacy of learning and reduce the psychological burden, but also change the angle of different distances and positions by the extension distance, increase the monitoring angle, and improve efficiency. At the same time, the output shaft of the second motor 3005 drives the screw 3002 to rotate, and the screw 3002 is threadedly connected to the moving block 3003, thereby driving the camera 3 to move position, further monitoring different positions, and increasing the coverage and accuracy of monitoring.

Claims

1. An intelligent environmental monitoring system based on image analysis, characterized in that: Include: Collect dormitory environment data, dormitory hygiene conditions and lights-off status data; Dormitory environment data preprocessing, including transcoding and cropping of dormitory environment data; Establish dormitory environment model training and use dormitory environment data to establish dormitory environment model; Dormitory environment model evaluation and adjustment: Use a portion of the reserved dormitory environment data to evaluate the trained model and make adjustments based on the results; Dormitory environment model deployment and application, deploying the trained and evaluated model into actual monitoring; The execution device makes further movement and approach according to the feedback result; The execution device comprises a fixing frame, which is fixedly arranged in the middle of a dormitory with multiple rooms; and a camera, which is arranged outside the fixing frame through an extension component and is moved left and right through a moving component.

2. The environment intelligent monitoring system based on image analysis according to claim 1 is characterized in that: The dormitory environment data is collected by using a camera to obtain image data, including the hygiene and tidying conditions of each location in the dormitory, and the status data of the lights-out time and each bed after lights-out is obtained.

3. The environment intelligent monitoring system based on image analysis according to claim 1 is characterized in that: The dormitory environment data is preprocessed by labeling the dormitory environment data to distinguish different hygiene states and states after lights out. For multi-classification problems, one-hot encoding is used to map each category into a vector in which only one element is 1 and the rest are 0; Detect and process missing or damaged data that may exist in the image. When the value of a pixel is missing, linear interpolation uses the values ​​of its adjacent pixels for estimation. Assuming that in the horizontal direction, the left and right adjacent pixel values ​​of the missing pixel are and respectively, the value of the pixel can be calculated by linear interpolation.

4. The environment intelligent monitoring system based on image analysis according to claim 1 is characterized in that: The dormitory environment model training is established, a convolutional neural network framework is selected, and the training enables it to accurately identify the dormitory hygiene conditions or the lights-off status; The dormitory environment data is divided into a training set, a data set used to train the dormitory environment model, and the dormitory environment model learns features and patterns through the training set.

5. The environment intelligent monitoring system based on image analysis according to claim 1 is characterized in that: The dormitory environment model is evaluated and adjusted using evaluation indicators for precision, which is the proportion of samples that are truly positive among samples predicted to be positive; the recall model is the proportion of samples that are successfully predicted to be positive among samples that are actually positive; the harmonic mean of precision and recall is used to comprehensively consider the two.

6. The environment intelligent monitoring system based on image analysis according to claim 1 is characterized in that: A stop door is hinged on the outside of the fixed frame, and the stop door is arranged in a double-leaf configuration; a pulling cavity is opened on the outside of the fixed frame, and a spring is fixedly arranged inside the pulling cavity, and the outer end of the spring is connected and fixed to the inner wall of the stop door; the spring is arranged in an inclined structure.

7. The environment intelligent monitoring system based on image analysis according to claim 6 is characterized in that: The extension assembly comprises a first telescopic block slidably arranged inside the fixing frame, a first screw rod is rotatably arranged inside the fixing frame, and the first screw rod is threadedly connected with a first threaded hole opened on the outer wall of the first telescopic block.

8. The environment intelligent monitoring system based on image analysis according to claim 7 is characterized in that: A face gear is fixedly provided at one end of the first screw rod, and a rotating gear is meshedly connected to the outer wall of the face gear. An engine cavity is provided inside the fixed frame, and a first motor is provided inside the engine cavity. The output shaft of the first motor is coaxially and fixedly connected to the rotating gear.

Images