Monitoring method and system based on Internet of Things

Abstract

The invention discloses a monitoring method and system based on the Internet of Things. The monitoring method based on the Internet of Things comprises the following steps: acquiring a monitoring image; preprocessing the monitoring image to obtain a target image; performing non-target monitoring according to the target image, and judging whether a non-target exists or not; and if the non-target does not exist, performing target state monitoring, and outputting a monitoring result. According to the method, the monitoring process can be monitored, so that whether the monitoring process is abnormal or not is judged, and the monitoring accuracy can be improved on the basis of saving the comparison time by performing multi-aspect and all-directional comparison on the monitored video images.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular, to a monitoring method and system based on the Internet of Things. Background Art

[0002] The Internet of Things (IoT) is a network technology that connects objects to the Internet through a variety of sensor devices to achieve the intelligence of objects. The IoT can be used to monitor or supervise the target object or environment to generate corresponding surveillance videos. However, in the monitoring process, since the surveillance video has a certain length, there may be monitoring anomalies that are not discovered in time. Traditionally, the normal monitoring is usually ensured by equipment maintenance and troubleshooting, and regular maintenance of surveillance cameras, storage devices, etc., such as cleaning camera lenses, checking whether the device connection is loose, etc., to ensure the normal operation of the equipment. However, the above method cannot detect the anomalies of the monitoring process in time during monitoring, and thus cannot guarantee the accuracy of the monitoring process.

[0003] Therefore, how to provide a more timely and accurate method for monitoring the monitoring process has become an urgent problem to be solved in this field. Summary of the invention

[0004] The present application proposes a monitoring method based on the Internet of Things, comprising the following steps: acquiring a monitoring image; preprocessing the monitoring image to obtain a target image; performing non-target monitoring based on the target image to determine whether there is a non-target; if there is no non-target, performing target status monitoring and outputting the monitoring results.

[0005] In the monitoring method based on the Internet of Things as described above, a monitoring image is obtained through the Internet of Things, and the monitoring image is an image containing a target.

[0006] In the monitoring method based on the Internet of Things as described above, preprocessing the monitoring image to obtain the target image includes dividing the acquired monitoring image into multiple frames, each frame being the target image.

[0007] As described above, the monitoring method based on the Internet of Things, wherein non-target monitoring is performed based on the target image, includes the following sub-steps: determining a starting target frame image in the target image; performing frame image comparison based on the starting target frame image to determine whether there is a non-target.

[0008] In the monitoring method based on the Internet of Things as described above, determining whether there are non-target objects includes whether there are other non-target pixels except the target image pixels; if there are other non-target pixels except the target image pixels, then there are non-target objects, the monitoring process is normal, and the process exits.

[0009] A monitoring system based on the Internet of Things specifically includes: an acquisition unit, a preprocessing unit, a non-target monitoring unit and a target state monitoring unit; the acquisition unit is used to acquire a monitoring image; the preprocessing unit is used to preprocess the monitoring image to obtain a target image; the non-target monitoring unit is used to perform non-target monitoring based on the target image to determine whether a non-target exists; if no non-target exists, the target state monitoring unit performs target state monitoring and outputs a monitoring result.

[0010] In the monitoring system based on the Internet of Things as described above, the acquisition unit acquires a monitoring image through the Internet of Things, and the monitoring image is an image containing a target.

[0011] In the monitoring system based on the Internet of Things as described above, the preprocessing unit preprocesses the monitoring image to obtain the target image, including dividing the acquired monitoring image into multiple frames, each frame being the target image.

[0012] In the monitoring system based on the Internet of Things as described above, the non-target monitoring unit performs non-target monitoring according to the target image, including the following sub-steps: determining a starting target frame image in the target image; performing frame image comparison according to the starting target frame image to determine whether there is a non-target.

[0013] In the monitoring system based on the Internet of Things as described above, the non-target monitoring unit determines whether there are non-target objects, including whether there are other non-target pixels except the target image pixels; if there are other non-target pixels except the target image pixels, then there is a non-target, the monitoring process is normal, and the process exits.

[0014] This application has the following beneficial effects:

[0015] The present application can monitor the monitoring process to determine whether there is any abnormality in the monitoring process. By performing a comprehensive comparison of the video images in the monitoring, it can save the comparison time and improve the accuracy of the monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0017] Figure 1 is a flow chart of a monitoring method based on the Internet of Things provided according to an embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of the internal structure of a monitoring system based on the Internet of Things provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0020] The present application proposes a monitoring method and system based on the Internet of Things, which can monitor the monitoring process and determine whether an abnormality occurs in the monitoring process, thereby ensuring the normal operation of the monitoring process.

[0021] Embodiment 1

[0022] like Figure 1 As shown, this embodiment provides a monitoring method based on the Internet of Things, which specifically includes the following steps:

[0023] Step S1: Acquire monitoring images.

[0024] The Internet of Things can be used to monitor the target and obtain the corresponding monitoring video. The monitoring video can be understood as the monitoring image corresponding to the target.

[0025] The target may be a moving person, and the surveillance video corresponding to the target is a moving video of the human body.

[0026] Step S2: pre-process the monitoring image to obtain a target image.

[0027] The acquired monitoring image is divided into multiple frame images, each frame image is a target image.

[0028] Specifically, multiple time nodes are selected to obtain multiple frames of surveillance video, each time node corresponds to a frame of image, and each frame of image reflects the movement of the target at the corresponding time node.

[0029] Step S3: Perform non-target monitoring based on the target image.

[0030] The non-target monitoring is to monitor images other than the target image. Through the non-target monitoring, it can be determined whether there are objects other than the target in the target image.

[0031] For example, if the surveillance video is of a person walking, then non-target monitoring can be used to determine whether other objects, such as pets or steps, appear during the person's walking process. This can be used to determine that the monitoring is continuous. Through this determination, it can be determined that the monitoring process is running normally without any abnormalities, and the walking process can be monitored for a continuous period of time. On the contrary, if no other objects other than people are monitored during the continuous period of time, then this embodiment believes that the monitoring process may be abnormal, resulting in interruption or freezing of the surveillance video.

[0032] Based on the above, step S3 includes the following sub-steps:

[0033] Step S31: Determine a starting target frame image in the target image.

[0034] The frame image corresponding to the initial time node among the multiple time nodes is used as the starting target frame image.

[0035] Step S32: performing frame image comparison based on the initial target frame image to determine whether there is a non-target.

[0036] The starting target frame image is compared with the frame images corresponding to the adjacent time nodes in the order of the time nodes (defined as the first comparison) to determine whether there is a non-target object.

[0037] The criterion for determining whether there are non-target objects is whether there are other non-target pixels besides the target image pixels. If there are other non-target pixels besides the target image pixels, it is considered that there are non-target objects, and there is no need to continue comparing the remaining frame images. It is considered that the monitoring is normal at this time, and the process exits.

[0038] If no non-target object is found after the first comparison, the starting target frame image is compared with the frame image corresponding to the closest time node in the order of time nodes to determine whether there is a non-target object.

[0039] The method of comparing the starting target frame image with the frame image each time is consistent with the above. After determining that a non-target object exists, the monitoring process is considered normal and the process exits. Otherwise, the starting target frame image continues to be compared with the frame images corresponding to the remaining time nodes. If the starting target frame image is compared with all the frame images and it is determined that there is no non-target object, step S4 is executed.

[0040] It is worth noting that each time the comparison is performed, the starting frame image is compared with the frame image corresponding to a single time node. After determining that the comparison needs to continue, the time node is excluded, and the closest time node among the remaining time nodes is selected for comparison with the frame image corresponding to the time node.

[0041] This embodiment determines non-target objects by frame-by-frame comparison. It only needs to determine that the monitoring process is normal if the pixels of the non-target objects appear in one frame of image, which can save the time cost of comparison to the greatest extent.

[0042] The comparison between the starting target frame image and the frame image may be performed by using image histogram comparison, or by using an image comparison tool to complete the comparison between the images.

[0043] Furthermore, in order to ensure the accuracy of the comparison process, this embodiment further includes performing a filtering operation on the starting target frame image and the frame image to be compared before the starting target frame image is compared with the frame image.

[0044] The Gaussian low-pass filtering algorithm is used to filter the starting target frame image and the frame image to be compared, filter out high-frequency noise, and retain the detailed features of the starting target frame image and the frame image to be compared, thereby ensuring the accuracy of subsequent comparison.

[0045] The specific filtering method is to use an n times n dimensional (i.e. n rows and n columns) window containing weighted coefficients to place on the starting frame image (or the frame image to be compared), and perform weighted averaging on the starting frame image (or the frame image to be compared). The value of each pixel is obtained by weighted averaging the value of itself and other pixels in the neighborhood of the window. In summary, the calculation formula for the pixel value of the starting target frame image (or the frame image to be compared) after filtering is as follows:

[0046]

[0047] Among them, k(x,y) represents the pixel value of the pixel with the coordinates (x,y) in the starting target frame image (or the frame image to be compared) after filtering, r(x,y) represents the pixel value of the pixel with the coordinates (x,y) in the starting target frame image (or the frame image to be compared), (x,y) represents the coordinates of the pixel point, σ represents the variance, the larger σ is, the wider the frequency band is, and the better it can suppress noise; represents the convolution operation; e represents a constant, e=2.718, and n represents the dimension of the filter window.

[0048] By filtering the starting target frame image and the frame image to be compared in the above manner, noise of the two is suppressed, thereby improving the accuracy of subsequent comparison.

[0049] Step S4: Perform target status monitoring and output monitoring results.

[0050] In the above step S3, by comparing the two frames of images, if it is considered that no objects other than the target can be obtained in the collected video data, it is considered that the monitoring process is abnormal. If objects other than the target can be obtained, it is necessary to further judge the state of the target and finally reach a conclusion on whether the monitoring video is abnormal.

[0051] Specifically, a reference image is first determined, the state of the reference image is used as the target starting state, and the state of the target is detected according to the reference image, so as to determine whether the target image moves during the monitoring process.

[0052] For example, if the surveillance video is of a person walking, the state of the person walking can be obtained through the video, for example, the person swings his arms while walking. Then the swinging left hand at the beginning can be used as the reference image.

[0053] If the monitored image is the same or similar to the reference image, if it is the same or similar, it is considered that the target has not changed its state. If the target's state has not changed within the duration, it is considered that the monitoring process is abnormal. If the target's state has changed within the duration, it is considered that the monitoring process is normal.

[0054] Based on the above, step S4 includes the following sub-steps:

[0055] Step S41: Determine a reference image.

[0056] The reference image is an image that only contains the target.

[0057] Step S42: Determine the area image where the target is located in the monitoring image.

[0058] The target area image is the image that only contains the target in the monitoring image.

[0059] Step S43: Determine the similarity between the reference image and the image of the target area.

[0060] The similarity T between the reference image and the target area image is expressed as:

[0061]

[0062] Where L represents the number of pixels in the reference image, J represents the number of pixels in the target area image, and Z i (x, y) represents the pixel value of the i-th pixel with coordinates (x, y) in the reference image, u j (x, y) represents the pixel value of the i-th pixel with coordinates (x, y) in the target area image, θ represents the diffraction limit value of the lens that actually collects the monitoring image; F -1 (·) represents the inverse Fourier operation.

[0063] If the similarity T is greater than the specified threshold, the monitored image is considered to be the same or similar to the reference image, and the state of the target is considered to have not changed, and the abnormal result of the monitoring process is output; if the similarity T is less than the specified threshold, the monitored image is considered to be not the same or similar to the reference image, and there are certain differences, and the normal result of the monitoring process is output.

[0064] Embodiment 2

[0065] like Figure 2 As shown, this embodiment provides a monitoring system based on the Internet of Things, which specifically includes: an acquisition unit 210, a preprocessing unit 220, a non-target monitoring unit 230 and a target state monitoring unit 240.

[0066] The acquisition unit 210 is used to acquire a surveillance image.

[0067] The target can be monitored through the Internet of Things to obtain the corresponding monitoring video, which can be understood as the monitoring image corresponding to the target. The target can be a moving person. The monitoring video corresponding to the target is a moving video of the human body.

[0068] The preprocessing unit 220 is used to preprocess the monitoring image to obtain a target image.

[0069] The acquired monitoring image is divided into multiple frame images, each frame image is a target image.

[0070] Specifically, multiple time nodes are selected to obtain multiple frames of surveillance video, each time node corresponds to a frame of image, and each frame of image reflects the movement of the target at the corresponding time node.

[0071] The non-target monitoring unit 230 is used to perform non-target monitoring according to the target image.

[0072] The non-target monitoring is to monitor images other than the target image. Through the non-target monitoring, it can be determined whether there are objects other than the target in the target image.

[0073] For example, if the surveillance video is of a person walking, then non-target monitoring can be used to determine whether other objects, such as pets or steps, appear during the person's walking process. This can be used to determine that the monitoring is continuous. Through this determination, it can be determined that the monitoring process is running normally without any abnormalities, and the walking process can be monitored for a continuous period of time. On the contrary, if no other objects other than people are monitored during the continuous period of time, then this embodiment believes that the monitoring process may be abnormal, resulting in interruption or freezing of the surveillance video.

[0074] Based on the above, the non-target monitoring unit 230 performs the following sub-steps:

[0075] Step Q1: Determine a starting target frame image in the target image.

[0076] The frame image corresponding to the initial time node among the multiple time nodes is used as the starting target frame image.

[0077] Step Q2: Perform frame image comparison based on the starting target frame image to determine whether there is a non-target.

[0078] The starting target frame image is compared with the frame images corresponding to the adjacent time nodes in the order of the time nodes (defined as the first comparison) to determine whether there is a non-target object.

[0079] The criterion for determining whether there are non-target objects is whether there are other non-target pixels besides the target image pixels. If there are other non-target pixels besides the target image pixels, it is considered that there are non-target objects, and there is no need to continue comparing the remaining frame images. It is considered that the monitoring is normal at this time, and the process exits.

[0080] If no non-target object is found after the first comparison, the starting target frame image is compared with the frame image corresponding to the closest time node in the order of time nodes to determine whether there is a non-target object.

[0081] The method of comparing the starting target frame image with the frame image each time is consistent with the above. After determining that a non-target object exists, the monitoring process is considered normal and the process exits. Otherwise, the starting target frame image continues to be compared with the frame images corresponding to the remaining time nodes. If the starting target frame image is compared with all the frame images and it is determined that there is no non-target object, step S4 is executed.

[0082] It is worth noting that each time the comparison is performed, the starting frame image is compared with the frame image corresponding to a single time node. After determining that the comparison needs to continue, the time node is excluded, and the closest time node among the remaining time nodes is selected for comparison with the frame image corresponding to the time node.

[0083] This embodiment determines non-target objects by frame-by-frame comparison. It only needs to determine that the monitoring process is normal if the pixels of the non-target objects appear in one frame of image, which can save the time cost of comparison to the greatest extent.

[0084] The comparison between the starting target frame image and the frame image may be performed by using image histogram comparison, or by using an image comparison tool to complete the comparison between the images.

[0085] Furthermore, in order to ensure the accuracy of the comparison process, this embodiment further includes performing a filtering operation on the starting target frame image and the frame image to be compared before the starting target frame image is compared with the frame image.

[0086] The Gaussian low-pass filtering algorithm is used to filter the starting target frame image and the frame image to be compared, filter out high-frequency noise, and retain the detailed features of the starting target frame image and the frame image to be compared, thereby ensuring the accuracy of subsequent comparison.

[0087] The specific filtering method is to use an n times n dimensional (i.e. n rows and n columns) window containing weighted coefficients to place on the starting frame image (or the frame image to be compared), and perform weighted averaging on the starting frame image (or the frame image to be compared). The value of each pixel is obtained by weighted averaging the value of itself and other pixels in the neighborhood of the window. In summary, the calculation formula for the pixel value of the starting target frame image (or the frame image to be compared) after filtering is as follows:

[0088]

[0089] Among them, k(x,y) represents the pixel value of the pixel with the coordinates (x,y) in the starting target frame image (or the frame image to be compared) after filtering, r(x,y) represents the pixel value of the pixel with the coordinates (x,y) in the starting target frame image (or the frame image to be compared), (x,y) represents the coordinates of the pixel point, σ represents the variance, the larger σ is, the wider the frequency band is, and the better it can suppress noise; represents the convolution operation; e represents a constant, e=2.718, and n represents the dimension of the filter window.

[0090] By filtering the starting target frame image and the frame image to be compared in the above manner, noise of the two is suppressed, thereby improving the accuracy of subsequent comparison.

[0091] The target state monitoring unit 240 is used to perform target state monitoring and output monitoring results.

[0092] First, a reference image is determined, and the state of the reference image is used as the target starting state. The state of the target is detected according to the reference image, so as to determine whether the target image moves during the monitoring process.

[0093] For example, if the surveillance video is of a person walking, the state of the person walking can be obtained through the video, for example, the person swings his arms while walking. Then the swinging left hand at the beginning can be used as the reference image.

[0094] If the monitored image is the same or similar to the reference image, if it is the same or similar, it is considered that the target has not changed its state. If the target's state has not changed within the duration, it is considered that the monitoring process is abnormal. If the target's state has changed within the duration, it is considered that the monitoring process is normal.

[0095] Based on the above, the target state monitoring unit 240 performs the following sub-steps:

[0096] Step P1: Determine a reference image.

[0097] The reference image is an image that only contains the target.

[0098] Step P2: Determine the area image where the target is located in the monitoring image.

[0099] The target area image is the image that only contains the target in the monitoring image.

[0100] Step P3: Determine the similarity between the reference image and the image of the target area.

[0101] The similarity T between the reference image and the target area image is expressed as:

[0102]

[0103] Where L represents the number of pixels in the reference image, J represents the number of pixels in the target area image, and Z i (x, y) represents the pixel value of the i-th pixel with coordinates (x, y) in the reference image, u j (x, y) represents the pixel value of the i-th pixel with coordinates (x, y) in the target area image, θ represents the diffraction limit value of the lens that actually collects the monitoring image; F -1 (·) represents the inverse Fourier operation.

[0104] If the similarity T is greater than the specified threshold, the monitored image is considered to be the same or similar to the reference image, and the state of the target is considered to have not changed, and the abnormal result of the monitoring process is output; if the similarity T is less than the specified threshold, the monitored image is considered to be not the same or similar to the reference image, and there are certain differences, and the normal result of the monitoring process is output.

[0105] The present application also provides a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the monitoring method based on the Internet of Things.

[0106] The embodiment disclosed in the present invention provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer executes the above-mentioned monitoring method based on the Internet of Things.

[0107] An embodiment of the present invention provides a processor for processing the above-mentioned monitoring method based on the Internet of Things.

[0108] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0109] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0110] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0111] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM).

[0112] This application has the following beneficial effects:

[0113] The present application can monitor the monitoring process to determine whether there is any abnormality in the monitoring process. By performing a comprehensive comparison of the video images in the monitoring, it can save the comparison time and improve the accuracy of the monitoring.

[0114] Although the present application is described with reference to examples, this is for illustrative purposes only and is not intended to limit the present application, and changes, additions and / or deletions to the embodiments may be made without departing from the scope of the present application.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A monitoring method based on the Internet of Things, characterized in that: The following steps are involved: Acquire surveillance images; Preprocess the monitoring image to obtain the target image; Perform non-target monitoring based on the target image to determine whether there is a non-target; If there is no non-target, the target status is monitored and the monitoring results are output.

2. The monitoring method based on the Internet of Things as claimed in claim 1, characterized in that: The monitoring image is obtained through the Internet of Things, and the monitoring image is an image containing a target.

3. The monitoring method based on the Internet of Things as claimed in claim 2, characterized in that: Preprocessing the monitoring image to obtain the target image includes dividing the acquired monitoring image into multiple frame images, each frame image is the target image.

4. The monitoring method based on the Internet of Things as claimed in claim 3, characterized in that: Non-target monitoring based on target images includes the following sub-steps: Determine a starting target frame image in the target image; Perform frame image comparison based on the starting target frame image to determine whether there is a non-target.

5. The monitoring method based on the Internet of Things as claimed in claim 4, characterized in that: Determining whether there is a non-target object includes determining whether there are other non-target pixels other than the target image pixels; If there are other non-target pixels besides the target image pixels, then there are non-targets, the monitoring process is normal, and the process exits.

6. A monitoring system based on the Internet of Things, characterized in that: Specifically include: An acquisition unit, a preprocessing unit, a non-target monitoring unit, and a target state monitoring unit; An acquisition unit, used for acquiring monitoring images; A preprocessing unit, used for preprocessing the monitoring image to obtain a target image; A non-target monitoring unit is used to perform non-target monitoring based on the target image and determine whether there is a non-target; If there is no non-target, the target state monitoring unit performs target state monitoring and outputs the monitoring result.

7. The monitoring system based on the Internet of Things as claimed in claim 6, characterized in that: The acquisition unit acquires a monitoring image through the Internet of Things, and the monitoring image is an image containing a target.

8. The monitoring system based on the Internet of Things as claimed in claim 7, characterized in that: The preprocessing unit preprocesses the monitoring image to obtain the target image, including dividing the acquired monitoring image into multiple frame images, each frame image is the target image.

9. The monitoring system based on the Internet of Things as claimed in claim 8, characterized in that: The non-target monitoring unit performs non-target monitoring according to the target image, including the following sub-steps: Determine a starting target frame image in the target image; Perform frame image comparison based on the starting target frame image to determine whether there is a non-target.

10. The monitoring system based on the Internet of Things as claimed in claim 9, characterized in that: The non-target monitoring unit determines whether there is a non-target object, including whether there are other non-target pixels except the target image pixels; If there are other non-target pixels besides the target image pixels, then there are non-targets, the monitoring process is normal, and the process exits.

Images