Field monitoring image transmission method

Abstract

The invention relates to a field monitoring image transmission method, the method is executed by an Internet of Things terminal, and the method comprises the following steps: obtaining a monitoring video transmitted by any field monitoring device after any startup, the monitoring video being composed of a plurality of monitoring images; determining a basic offset degree of the monitoring video; determining a shear value of each image according to the basic offset degree; cutting the monitoring video according to the cutting value; and after transmission triggering is obtained, the cut monitoring video is transmitted. According to the method, the basic offset degree of the monitoring video is determined; determining a shear value of each image according to the basic offset degree; cutting the monitoring video according to the cutting value; and after the transmission trigger is obtained, the cut monitoring video is transmitted, so that repeated images can be effectively cut off, the data volume of the cut monitoring video is reduced, and waste of transmission resources and storage resources is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of image transmission, and in particular to a field monitoring image transmission method. Background Art

[0002] As human beings continue to destroy the natural environment, the population of wild animals has decreased sharply. Through field monitoring, we can obtain original data such as the survival of wild animals, which is helpful for the analysis and protection of wild animal populations.

[0003] Currently, field monitoring equipment (such as cameras or infrared cameras) will automatically identify power-on triggers. When the power-on trigger is identified, it will automatically start up to obtain monitoring videos and transmit the monitoring videos to the IoT terminal. After the IoT terminal aggregates the monitoring videos, it will send them to the IoT forwarding node (such as a drone), which will then be transmitted back to the IoT service node (such as a server) by the intermediate node.

[0004] In actual practice, field monitoring equipment may have problems such as power-on trigger misidentification, which will result in duplicate images in the collected video. These images will not only increase the amount of monitoring video data and waste transmission resources, but also consume unnecessary storage resources. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In order to solve the above problems, the present invention provides a field monitoring image transmission method.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] A method for transmitting field monitoring images, the method being executed by an Internet of Things terminal, the method comprising:

[0010] Obtain the surveillance video transmitted by any field surveillance equipment after any startup, the surveillance video consisting of multiple surveillance images;

[0011] Determine the basic deviation of the surveillance video;

[0012] Determine the shear value of each image according to the basic offset;

[0013] Cut the surveillance video according to the cutting value;

[0014] When the transmission trigger is obtained, the cut surveillance video is transmitted.

[0015] Optionally, determining a basic deviation of the surveillance video includes:

[0016] Get the first image of the surveillance video;

[0017] Get standard images from any field monitoring equipment;

[0018] Performing object recognition on the first image to determine a first set of pixels involved in each object;

[0019] Performing object recognition on the standard image to determine a second set of pixels involved in each object;

[0020] Determine a first grayscale value of each pixel in the grayscale image of the first image;

[0021] Determining a second grayscale value of each pixel in the grayscale image of the standard image;

[0022] For any pixel in the first image, the grayscale value deviation of any pixel is determined as the absolute value of the difference between the first grayscale value of any pixel and the second grayscale value of any pixel; the object quantity deviation of any pixel is determined as the absolute value of the difference between the first set quantity containing any pixel and the second set quantity containing any pixel; the object overall deviation of any pixel is determined as the absolute value of the difference between the total number of elements in the union of all the first set quantities containing any pixel and the total number of elements in the union of all the second set quantities containing any pixel;

[0023] Determine the offset value of each pixel in the first image according to the gray value deviation of each pixel, the object quantity deviation and the object overall deviation;

[0024] The basic offset of the surveillance video is determined as the number of pixels in the first image whose offset value is less than a preset offset threshold / the total number of pixels in the first image.

[0025] Optionally, determining the offset value of each pixel in the first image according to the gray value deviation of each pixel, the object quantity deviation and the object overall deviation includes:

[0026] The offset value for any pixel in the first image is e^(grayscale value deviation of any pixel / maximum value of grayscale value deviation of all pixels in the first image + object quantity deviation of any pixel / maximum value of object quantity deviation of all pixels in the first image + overall object deviation of any pixel / maximum value of overall object deviation of all pixels in the first image).

[0027] Optionally, determining the shearing value of each image according to the basic offset includes:

[0028] Determine the comparison image of each image according to the basic offset;

[0029] Determine each object in each image and its minimum bounding rectangle;

[0030] Determine each object and its minimum bounding rectangle in the comparison images of each object;

[0031] For any image, if the object in any image is the same as the object in its comparison image, then determine the overlapping area between the minimum bounding rectangle of each object in any image and the minimum bounding rectangle of the corresponding object in its comparison image;

[0032] Determine the offset ratio of each object in either image based on the overlapped area;

[0033] If the maximum value of the offset ratio of objects of all target types in any image is less than a preset offset ratio threshold, the maximum value of the offset ratio of objects of all target types in any image is determined to be the shearing value of any image; otherwise, the shearing value of any image is determined to be 1.

[0034] Optionally, determining an offset ratio of each object in any one of the images based on the overlapped area includes:

[0035] The offset ratio of each object in any image is determined as 1-the ratio of the number of pixels in the overlapped area of ​​the object to the number of pixels in the minimum bounding rectangle of the object in any image.

[0036] Optionally, determining a comparison image of each image according to the basic offset includes:

[0037] For the first image, if the basic offset is less than the preset offset ratio, and the distribution of pixels in the first image whose offset value is less than the preset offset threshold conforms to the preset distribution, the standard image is determined as its comparison image, otherwise the first image is used as its comparison image;

[0038] For an image other than the first one, its previous image is determined as its comparison image.

[0039] Optionally, after determining each object and its minimum bounding rectangle in the comparison image of each object, the method further includes:

[0040] For any image, if the object in any image is different from the object in its comparison image, the type of the different object is determined;

[0041] If the type includes the target type, the shear value of any image is determined to be 1;

[0042] If the type does not include the target type, the shear value of any image is determined as the number of pixels in the minimum bounding rectangle of different objects / the total number of pixels in any image.

[0043] Optionally, the surveillance video is cut according to the cut value, including:

[0044] For each image, if its clipping value is not less than a preset clipping threshold, it is determined to be a retained image;

[0045] If the shearing value is less than the preset shearing threshold, a square window is constructed with a preset width, starting from the upper left corner of each image and the comparison image of each image, the upper left corner pixel is used as the upper left corner position of the square window, and the square window is moved from left to right and from top to bottom with a step length of 1 pixel, and the variance of the grayscale values ​​of all pixels in the square window is determined each time; the ratio of the mean of all variances in each image to the mean of all variances in its comparison image is determined as the average change value of the image; the ratio of the standard deviation of all variances in the image to the standard deviation of all variances in its comparison image is determined as the change degree value of the image; and whether it is a retained image is determined according to the average change value and the change degree value;

[0046] Cut out the cut image from the surveillance video.

[0047] Optionally, determining whether it is a retained image according to the average change value and the change degree value includes:

[0048] If the average change value is greater than 1, update it to the average change value - 1;

[0049] If the change degree value is greater than 1, it is updated to the change degree value - 1;

[0050] If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

[0051] Optionally, determining whether it is a retained image according to the average change value and the change degree value includes:

[0052] If the average change value is greater than 1, update it to the average change value - 1;

[0053] If the change degree value is greater than 1, it is updated to the change degree value - 1;

[0054] If the quotient of the current average change value and the current change degree is not less than the preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

[0055] (III) Beneficial effects

[0056] The present invention relates to a method for transmitting field monitoring images, which is executed by an Internet of Things terminal, and comprises: obtaining a monitoring video transmitted by any field monitoring device after any startup, wherein the monitoring video is composed of a plurality of monitoring images; determining a basic offset of the monitoring video; determining a shearing value of each image according to the basic offset; shearing the monitoring video according to the shearing value; and transmitting the sheared monitoring video after obtaining a transmission trigger. The method of the present invention determines a basic offset of the monitoring video; determines a shearing value of each image according to the basic offset; shearing the monitoring video according to the shearing value; and transmitting the sheared monitoring video after obtaining a transmission trigger, which can effectively shear off duplicate images, reduce the amount of data of the sheared monitoring video, and avoid waste of transmission resources and storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of a field monitoring image transmission method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0059] At present, field monitoring equipment (such as cameras or infrared cameras) will automatically identify the power-on trigger. When the power-on trigger is identified, it will automatically start to obtain the monitoring video and transmit the monitoring video to the IoT terminal. After the IoT terminal summarizes the monitoring video, it will send it to the IoT forwarding node (such as a drone), and then the intermediate node will transmit it back to the IoT service node (such as a server). In actual practice, field monitoring equipment may have problems such as incorrect recognition of the power-on trigger, which will result in duplicate images in the collected video. These images will not only increase the amount of monitoring video data, waste transmission resources, but also consume unnecessary storage resources.

[0060] To this end, the present invention relates to a method for transmitting field monitoring images, which is executed by an Internet of Things terminal, and the method includes: obtaining a monitoring video transmitted by any field monitoring device after any startup, wherein the monitoring video is composed of multiple monitoring images; determining a basic offset of the monitoring video; determining a shearing value of each image according to the basic offset; shearing the monitoring video according to the shearing value; and transmitting the sheared monitoring video after obtaining a transmission trigger. The method of the present invention determines a basic offset of the monitoring video; determines a shearing value of each image according to the basic offset; shearing the monitoring video according to the shearing value; and transmitting the sheared monitoring video after obtaining a transmission trigger, which can effectively shear off duplicate images, reduce the amount of data of the sheared monitoring video, and avoid waste of transmission resources and storage resources.

[0061] The method is applied to IoT terminals.

[0062] During field monitoring, one or more field monitoring devices (such as cameras, infrared cameras, etc.) can be installed in the monitoring area (such as areas where wild animals often appear). Each field monitoring device will be connected to the IoT terminal based on its own pre-configured wireless communication module (such as a 2.4GHz wireless communication module, etc.). The field monitoring device and the IoT terminal form a local area network connection.

[0063] Each field monitoring device is equipped with an infrared pyroelectric sensor and a camera device (such as a camera). When a wild animal is near the field monitoring device (such as a wild animal is 10 meters away from the field monitoring device), its infrared pyroelectric sensor will sense the wild animal, trigger the camera device to start up and collect monitoring images, and the collected multiple monitoring images will be automatically transmitted to the Internet of Things terminal. After a period of time (such as 15 minutes) after the camera device is turned on, or if the infrared pyroelectric sensor does not sense wild animals for a period of time (such as 5 minutes, that is, the infrared pyroelectric sensor does not sense wild animals within 5 minutes), the camera device in the field monitoring device will automatically shut down, and then stop transmitting images to the Internet of Things terminal. Here, all images transmitted to the Internet of Things terminal from startup to shutdown constitute the monitoring video transmitted by the field monitoring device after the startup.

[0064] The IoT forwarding node (such as a drone) will fly to the monitoring area based on the data collection instructions. The IoT forwarding node will establish a data transmission link with the IoT terminal in the monitoring area, and then send instructions to the IoT terminal to trigger the transmission. After receiving the transmission trigger, the IoT terminal will transmit the surveillance video transmitted by the field monitoring equipment it has obtained to the IoT forwarding node, delete the surveillance video, and make the IoT terminal clear its storage space so that it can continue to obtain the surveillance video transmitted by the field monitoring equipment connected to it after subsequent startup.

[0065] After the IoT forwarding node obtains the monitoring image transmitted by the IoT terminal, it will transmit the monitoring image to the server for subsequent analysis.

[0066] The method provided in this embodiment is executed by the Internet of Things terminal, and is used to optimize the surveillance video after obtaining the surveillance video transmitted by any field surveillance equipment after any startup, cut out the repeated images therein, and then obtain the cut surveillance video. Since the cut surveillance video does not contain repeated images, the amount of data of the cut surveillance video is greatly reduced compared with the surveillance video before cutting. After receiving the transmission trigger, the cut surveillance video will be transmitted to the Internet of Things forwarding node, which will reduce the demand for transmission resources and storage resources and avoid the waste of transmission resources and storage resources.

[0067] See also Figure 1This embodiment provides a method for transmitting field monitoring images. The method is executed by an Internet of Things terminal. The implementation process of the method is as follows:

[0068] 101, obtaining the monitoring video transmitted by any field monitoring device after any startup.

[0069] Among them, the surveillance video is composed of multiple surveillance images.

[0070] The camera in any field monitoring device will turn on and collect images when wild animals approach it. The collected images (i.e., monitoring images) will be automatically transmitted to the IoT terminal. The image collection and transmission will continue until the field monitoring device turns off the camera. All monitoring images collected by the camera from the time it is turned on to the time it is turned off will be automatically transmitted to the IoT terminal. The IoT terminal will form a monitoring video with all the images it has obtained and temporarily store the monitoring video. When the monitoring video is cut or the cut video is transmitted, the monitoring video at this time will be deleted.

[0071] The Internet of Things terminal will cut it through the method provided in this embodiment to ensure that the cut video does not contain repeated surveillance images, and further ensure that the cut video does not contain redundant video.

[0072] The IoT terminal will then store the cut surveillance video. The cut surveillance video will only be deleted after transmission. Therefore, the cut surveillance video stored on the IoT terminal is the surveillance video that has not been transmitted back to the server through the IoT forwarding node. After the next transmission trigger is obtained, all the cut surveillance videos stored in it can be transmitted. All the cut surveillance videos stored will be deleted after transmission.

[0073] In step 101, the IoT terminal obtains the stored surveillance video transmitted by any field surveillance device after any startup. The surveillance video here is the original captured video and has not been cut, so there will be repeated surveillance images, that is, there will be image redundancy. The surveillance video obtained in step 101 will be cut through steps 102 to 104, and the repeated surveillance images (i.e. redundant images) in the surveillance video will be cut to ensure that the surveillance video after cutting does not contain redundant images, thereby reducing the data volume of the surveillance video after cutting and avoiding the waste of transmission resources and storage resources.

[0074] 102, determine the basic deviation of the surveillance video.

[0075] The deviation represents the difference between the monitoring video and the standard image captured by the field monitoring equipment.

[0076] After the field monitoring equipment is installed, a standard image is collected regularly (e.g., periodically, every day). The standard image is a normal (wild animal-free) image collected by the field monitoring equipment (e.g., a monitoring image is collected once a day at 8 o'clock, and the monitoring image shows the normal image among the monitoring images collected by the field monitoring equipment today, such as which plants are included, the basic color tone of the monitoring image caused by the weather, etc.). The change degree of the currently collected monitoring image can be obtained through the standard image, and the change degree is the basic deviation degree.

[0077] The implementation process of step 102 is as follows:

[0078] 102-1, obtain the first image of the surveillance video.

[0079] The first image is the first image transmitted by any field monitoring device after it is turned on.

[0080] 102-2, obtain a standard image of any field monitoring device.

[0081] For example, the standard image is the standard image collected and transmitted by the field monitoring equipment at the last 8 points.

[0082] 102-3, perform object recognition on the first image to determine a first set of pixels involved in each object.

[0083] This step will use the existing object recognition scheme in the image to perform object recognition on the first image, and obtain each object and the pixel identification covered by each object, which forms a set, namely the first set. It should be noted that the object here is a complete object, such as a panda eating bamboo leaves. For the bamboo leaves at its mouth, half are in the monitoring image and half are not (eaten into the panda's mouth). After identifying half of the bamboo leaves in the monitoring image, it will be completed through the existing AI solution. The completed object (including the uneaten part of the bamboo leaves and the part that has been eaten into the mouth) is the object described in this embodiment. Therefore, the object described in this embodiment must be a complete object, so there is a situation where one pixel point corresponds to multiple objects, such as the pixel of the panda's mouth, which corresponds to both the panda and the leaves.

[0084] If there are multiple objects in the first image (such as multiple leaves, multiple stones, multiple flowers), etc., each leaf, each stone, each flower, and the pixels covered by each leaf in the first image, the pixels covered by each stone in the first image, and the pixels covered by each flower in the first image will be identified in step 102-3. The pixels covered by each leaf in the first image form a first set of the leaf, the pixels covered by each stone in the first image form a first set of the stone, and the pixels covered by each flower in the first image form a first set of the flower. In this way, there will be as many first sets as the number of objects identified in step 102-3. Each first set corresponds to an object. At the same time, the object's identifier will also be identified (such as determining the type of the object, such as a leaf, etc., through an existing object recognition scheme in an image), and the type is the identifier of the object, and the identifier is used as an attribute of the first set.

[0085] 102-4, perform object recognition on the standard image to determine a second set of pixels involved in each object.

[0086] The implementation process of this step is similar to step 102-3. Please refer to the description of step 102-3 for details, which will not be elaborated here.

[0087] 102-5, determine a first grayscale value of each pixel in the grayscale image of the first image.

[0088] This step may adopt an existing grayscale image conversion solution to convert the first image into a grayscale image, and then obtain a first grayscale value of each pixel in the grayscale image of the first image.

[0089] 102-6, determine a second grayscale value of each pixel in the grayscale image of the standard image.

[0090] The implementation process of this step is similar to step 102-5. Please refer to the description of step 102-5 for details, which will not be elaborated here.

[0091] 102-7, for any pixel in the first image, determine the gray value deviation of any pixel as the absolute value of the difference between the first gray value of any pixel and the second gray value of any pixel. Determine the object quantity deviation of any pixel as the absolute value of the difference between the first set quantity containing any pixel and the second set quantity containing any pixel. Determine the object overall deviation of any pixel as the absolute value of the difference between the total number of elements in the union of all the first set quantities containing any pixel and the total number of elements in the union of all the second set quantities containing any pixel.

[0092] For example, for any pixel i, the first grayscale value of pixel i in the first image is G1(i), and the second grayscale value of pixel i in the standard image is G2(i), then the grayscale value deviation GD(i) of pixel i is determined to be |G1(i)-G2(i)|. The grayscale value deviation of any pixel represents the difference in grayscale value between the first image and the standard image for the same pixel. The larger the value, the greater the difference in the pixel, which means that the surveillance video collected this time is less the normal image of the environment.

[0093] For any pixel i, determine the number O1(i) of first sets containing pixel i in all first sets, and determine the number O2(i) of second sets containing pixel i in all second sets. Then determine the object quantity deviation OD(i) = |O1(i)-O2(i)| for pixel i. The object quantity deviation of any pixel represents the difference in the number of objects belonging to the same pixel. The larger the difference, the more different the object represented by the pixel (for example, pixel i in the first image is a pixel covered by leaves, but there is no object at pixel i in the standard image, and at this time there is a difference in the object at pixel i). The larger the value, the greater the difference of the pixel, which means that the surveillance video collected this time is less the normal image of the environment.

[0094] For any pixel i, in all first sets, all elements in the first set containing pixel i are combined (such as a panda eating bamboo leaves, the panda and the bamboo leaves overlap on pixel i, then the objects corresponding to pixel i are leaves and pandas, then the first set of leaves and the first set of pandas will be merged), and the number of elements in the set after the combination operation E1(i) is obtained, and E1(i) represents the range of all objects corresponding to pixel i. In all second sets, all elements in the second set containing pixel i are combined, and the number of elements in the set after the combination operation E2(i) is obtained. Then the overall object deviation ED(i) of pixel i is determined to be |E1(i)-E2(i)|. The overall object deviation of any pixel represents the difference in the range of objects corresponding to the same pixel point. The larger the value, the greater the change in the corresponding object, which means that the surveillance video collected this time is less the usual image of the environment.

[0095] 102-8, determining the offset value of each pixel in the first image according to the gray value deviation of each pixel, the object quantity deviation and the object overall deviation.

[0096] The offset value for any pixel in the first image is e^(grayscale value deviation of any pixel / maximum value of grayscale value deviation of all pixels in the first image + object quantity deviation of any pixel / maximum value of object quantity deviation of all pixels in the first image + overall object deviation of any pixel / maximum value of overall object deviation of all pixels in the first image).

[0097] For example, for any pixel i, the offset value D(i) of pixel i=e^(GD(i) / GD(i-max)+OD(i) / OD(i-max)+ED(i) / ED(i-max)).

[0098] Among them, GD(i-max) is the maximum value of the gray value deviation of all pixels in the first image, OD(i-max) is the maximum value of the object quantity deviation of all pixels in the first image, and ED(i-max) is the maximum value of the overall object deviation of all pixels in the first image.

[0099] GD(i) / GD(i-max) represents the gray value deviation ratio of pixel i in the first image, OD(i) / OD(i-max) represents the object quantity deviation ratio of pixel i in the first image, and ED(i) / ED(i-max) represents the overall object deviation ratio of pixel i in the first image. The offset value of any pixel describes the difference between the pixel and the standard image from three dimensions: the gray value of the pixel, the corresponding object, and the range of the corresponding object. The smaller the value, the smaller the difference, and the more the surveillance video collected this time is the normal image of the environment. The larger the value, the greater the difference, and the less the surveillance video collected this time is the normal image of the environment.

[0100] 102-9, determining the basic offset of the surveillance video as the number of pixels in the first image whose offset value is less than a preset offset threshold / the total number of pixels in the first image.

[0101] The offset threshold is an empirical value, which is a value between 0 and 1. The offset threshold is determined according to the transmission rate and accuracy requirements of the monitoring image. If fast transmission is required and very accurate monitoring images are not required, the offset threshold can be slightly larger, so that the number of pixels in the first image with offset values ​​less than the preset offset threshold increases, the basic offset degree increases, and the smaller the change between the first image and the standard image, the more similar they are. If the transmission rate is not limited, but accurate monitoring images are required, the offset threshold can be slightly smaller, so that the number of pixels in the first image with offset values ​​less than the preset offset threshold decreases, the basic offset degree decreases, and the larger the change between the first image and the standard image, the less similar they are.

[0102] If the offset value is less than the preset offset threshold, it means that the pixel has changed but it can be ignored and is considered unchanged. Otherwise, the pixel is considered changed. Therefore, the basic offset of the surveillance video is the ratio of pixels that have not changed between the first image and the standard image. The larger the value, the smaller the change between the first image and the standard image, and the more similar they are.

[0103] 103, determining the shearing value of each image according to the basic offset.

[0104] The shear value represents the repetition degree of each image, which is the repetition degree compared with a standard. Therefore, the implementation process of this step is:

[0105] 103-1, determining a comparison image of each image according to the basic offset.

[0106] The comparison image of any image is the standard of that image.

[0107] For the first image, if the basic offset is less than the preset offset ratio, and the pixel distribution of the offset value in the first image that is less than the preset offset threshold conforms to the preset distribution, the comparison image is determined as its basic image, otherwise the first image is used as its basic image. For non-first images, the previous image is determined as its basic image.

[0108] Among them, the pixel distribution can be obtained based on the existing distribution calculation method, such as determining the pixel distribution by the K function. In addition, the preset distribution is such as uniform distribution. The offset ratio is also a preset empirical value, which is a value from 0 to 1 and is determined based on the accuracy requirement of the monitoring image. If a very precise monitoring image is not required, the offset ratio can be slightly larger, and the possibility that the basic offset is less than the preset offset ratio increases, and the possibility of considering the first image to be similar to the comparison image increases, and the case of determining the comparison image as its basic image increases. If a precise monitoring image is required, the offset ratio can be slightly smaller, and the possibility that the basic offset is less than the preset offset ratio decreases, and the possibility of considering the first image to be similar to the comparison image decreases, and the case of using the first image as its basic image increases.

[0109] For the first image, if the basic offset is less than the preset offset ratio, it means that the first image is very similar to the standard image; if the distribution of pixels in the first image whose offset value is less than the preset offset threshold conforms to the preset uniform distribution, it means that the distribution of the same (not necessarily exactly the same, as long as it is less than the preset offset threshold) pixels in the first image and the standard image is relatively uniform, then it can be considered that the distribution of different pixels (that is, not less than the preset offset threshold) is also uniform. If wild animals appear, etc., the distribution of different pixels must be non-uniform. Therefore, if the basic offset is less than the preset offset ratio, and the distribution of pixels in the first image whose offset value is less than the preset offset threshold conforms to the preset distribution, then it is considered that the first image is very similar to the standard image, or even the same. At this time, the standard image can be determined as the basic image of the first image. Otherwise, it is considered that the first image is very different from the standard image, and it cannot be compared with the standard image at this time. Therefore, the first image will be used as the basic image of the first image itself, that is, it is its own basic image.

[0110] For a non-first image, it is necessary to determine whether it is the same as the previous image, so the previous image is determined as its base image.

[0111] In this way, we only need to compare the difference between each image and the base image to determine whether the image is a redundant image and whether it needs to be cut out.

[0112] 103-2, determine each object in each image and its minimum bounding rectangle.

[0113] The implementation process of object determination in this step is similar to step 102-3, see the description of step 102-3 for details, and will not be elaborated here. After obtaining the object, the minimum rectangle containing the object is determined as the minimum circumscribed rectangle. This process is also the basic process of object recognition and can be obtained by using existing object recognition related implementation solutions.

[0114] At the same time, the identity of the object is also identified.

[0115] 103-3, determine each object and its minimum circumscribed rectangle in the comparison image of each object.

[0116] The implementation process of this step is similar to step 103 - 2 , please refer to the description of step 103 - 2 for details, and will not be elaborated here.

[0117] In addition, since it is not the first image, its comparison image is the previous image, and the object in this image and its minimum enclosing rectangle have been determined in 103-2, and the determination result of step 103-2 can be directly obtained here. For the first image, if its comparison image is itself, then the object in this image and its minimum enclosing rectangle have been determined in 103-2, and the determination result of step 103-2 can be directly obtained here. If its comparison image is a standard image, then the object in this image has been identified in 102-4, and the result of step 102-4 can be directly obtained here. The minimum enclosing rectangle of each object can be determined in step 103-3 using existing methods.

[0118] At the same time, the identity of the object is also determined.

[0119] 103-4, for any image, if the object in any image is the same as the object in the comparison image, then execute steps 201 to 203. If the object in any image is not the same as the object in the comparison image, then execute steps 301 and 302.

[0120] Step 103-4 can determine whether the object in any image is the same as the object in its comparison image based on the identifier. If the identifier is the same, the object in any image is considered to be the same as the object in its comparison image. Otherwise, the object in any image is considered to be different from the object in its comparison image.

[0121] 201 : Determine an overlapping area between the minimum bounding rectangle of each object in any image and the minimum bounding rectangle of the corresponding object in the comparison image.

[0122] The overlap area represents the change of the area of ​​the same object in two consecutive surveillance images. The smaller the overlap area, the greater the movement of the object, which means the greater the change.

[0123] 202 , determining an offset ratio of each object in any one of the images based on the overlapped area.

[0124] The offset ratio of each object in any image is determined as 1-the ratio of the number of pixels in the overlapped area of ​​the object to the number of pixels in the minimum bounding rectangle of the object in any image.

[0125] For any image j, the number of pixels in the minimum bounding rectangle of any object u(j) in image j is NP(u(j)). The number of pixels in the overlapping area of ​​the minimum bounding rectangle of object u(j) in image j and the corresponding minimum bounding rectangle of object u(j) in the comparison image is CNP(u(j)), then the offset ratio of object u(j) in image j is OR(u(j)) = 1-CNP(u(j)) / NP(u(j)).

[0126] The ratio of the number of pixels in the overlapping area of ​​the object to the number of pixels in the minimum circumscribed rectangle of the object in any image represents the proportion of the overlapping area of ​​the object. The larger the value, the more similar the two are and the smaller the change is.

[0127] The offset ratio is 1-the ratio of the number of pixels in the overlapping area of ​​the object to the number of pixels in the smallest circumscribed rectangle of the object in any image. The smaller the offset ratio, the more similar the two are and the smaller the change. Therefore, the offset ratio represents the change of the object in two adjacent images. The larger the value, the greater the change.

[0128] 203, if the maximum value of the offset ratio of all target types of objects in any image is less than a preset offset ratio threshold, determine the maximum value of the offset ratio of all target types of objects in any image as the shearing value of any image. Otherwise, determine the shearing value of any image as 1.

[0129] The offset ratio threshold is an empirical value, which is a value between 0 and 1. The offset ratio threshold is determined according to the transmission rate and accuracy requirements of the surveillance image. If fast transmission is required and very accurate surveillance images are not required, the offset ratio threshold can be slightly larger, so the maximum value of the offset ratio of all target types of objects is less than the preset offset ratio threshold, and the number of images with a cut value of 1 is reduced, and the number of surveillance images cut off subsequently increases. If the transmission rate is not limited, but accurate surveillance images are required, the offset ratio threshold can be slightly smaller, so the maximum value of the offset ratio of all target types of objects is less than the preset offset ratio threshold, and the number of images with a cut value of 1 is increased, and the number of surveillance images cut off subsequently decreases.

[0130] In addition, the target type is also a preset type, which is determined based on the monitoring purpose. For example, if monitoring the survival of giant pandas in the wild, the target type is giant pandas. If monitoring the birth of bears in the wild, the target type is bears.

[0131] If the maximum value of the offset ratio of all target types of objects in any image is less than the preset offset ratio threshold, it means that the object of each target type has not changed much in the previous and next two images, then the maximum value of the offset ratio of all target types of objects in any image is determined to be the shear value of any image, so the shear value represents the change between two adjacent images, and the shear value is a value from 0 to 1, and the larger the value, the greater the change. If the maximum value of the offset ratio of at least one target type of object in any image is not less than the preset offset ratio threshold, it is considered that the change of the object is large, then the image is definitely different from the previous image and is not an easy image, so the shear value of any image is determined to be 1.

[0132] 301 , for any image, if an object in any image is different from an object in its comparison image, then determine the type of the different object.

[0133] This indicates that a new object appears in two consecutive images, or an object disappears, that is, there is an object change in any of the images (it may be that there are more objects in any of the images, or there may be less objects in any of the images, or there may be both more and less objects in any of the images).

[0134] 302, if the type includes the target type, it means that the changed object is of interest to the monitoring task, and therefore, the clipping value of any image is determined to be 1. If the type does not include the target type, it means that the changed object is not of interest to the monitoring task, and the clipping value of any image is determined to be the number of pixels of the minimum circumscribed rectangle of different objects / the total number of pixels in any image.

[0135] 104, cutting the surveillance video according to the cutting value.

[0136] The implementation process of step 104 is:

[0137] 104-1, for each image, if its cropping value is not less than a preset cropping threshold, it is determined to be a retained image.

[0138] The clipping threshold is an empirical value, which is a value between 0 and 1. The clipping threshold is determined according to the transmission rate and accuracy requirements of the monitoring image. If fast transmission is required and very accurate monitoring images are not required, the clipping threshold can be slightly larger, so that fewer images will be determined as retained images, and more images will be clipped. If the transmission rate is not limited, but accurate monitoring images are required, the clipping threshold can be slightly smaller, so that more images will be determined as retained images, and fewer images will be clipped.

[0139] 104-2, if the clipping value is less than the preset clipping threshold, a square window is constructed with a preset width, starting from the upper left corner of each image and the comparison image of each image, the upper left corner pixel is used as the upper left corner position of the square window, and the square window is moved from left to right and from top to bottom with a step length of 1 pixel, and the variance of the grayscale values ​​of all pixels in the square window is determined each time. The ratio of the mean of all variances in each image to the mean of all variances in its comparison image is determined as the average change value of the image. The ratio of the standard deviation of all variances in the image to the standard deviation of all variances in its comparison image is determined as the change degree value of the image. Determine whether it is a retained image based on the average change value and the change degree value.

[0140] That is to say, if its clipping value is less than the preset clipping threshold, it means that it may be a retained image or not, and the following steps are performed:

[0141] 1. Construct a square window with a preset width (such as establishing a 3×3 square window), starting from the upper left corner of each image and the comparison image of each image, use the upper left corner pixel as the upper left corner position of the square window, and move the square window from left to right and from top to bottom in a step length of 1 pixel (that is, starting from the upper left corner of the image, move the square window from left to right and from top to bottom, and move one pixel at a time), and determine the variance of the grayscale values ​​of all pixels in the square window each time (the variance represents the change in the grayscale values ​​of all pixels in the square window here).

[0142] 2. The ratio of the mean of all variances in each image (representing the mean of the change in grayscale value obtained by each pixel change in the image) to the mean of all variances in its comparison image (representing the mean of the change in grayscale value obtained by each pixel change in the comparison image of the image) is determined as the average change value of the image (the ratio represents the difference in the average change in grayscale value between the image and the comparison image, that is, the average change value).

[0143] The average change value represents the difference in pixel values ​​between the image and its comparison image as a whole. This value represents the difference in color between the two images and is a macroscopic change of the image as a whole.

[0144] 3. The ratio of the standard deviation of all variances in the image (characterizing the variation of the grayscale value variation obtained by each pixel change in the image, i.e., the variation of the grayscale value variation of each window) to the standard deviation of all variances in the comparison image (characterizing the variation of the grayscale value variation obtained by each pixel change in the comparison image, i.e., the variation of the grayscale value variation of each window) is determined as the variation degree value of the image (the ratio is the variation degree value, characterizing the difference in variation of the grayscale value variation of the image and the comparison image, i.e., the variation of the grayscale value variation of each window in the image compared to the variation of the grayscale value variation of each window in the comparison image).

[0145] The change degree value represents the degree of change in pixel values ​​between the pixels of the image and its comparison image as a whole. This value represents the change between the pixel values ​​of each pixel point in each window of the two images, and is a microscopic change based on each pixel in the image.

[0146] 4. Determine whether it is a retained image based on the average change value and the change degree value.

[0147] This can be achieved by the following steps.

[0148] 1) If the average change value is greater than 1, it means that the mean of the grayscale value change of the image is greater than the mean of the grayscale value change of its comparison image. Then it is updated to the average change value - 1, that is, the updated average change value reflects the change ratio between the two, that is, the difference between the mean of all variances in the image and the mean of all variances in its comparison image / the mean of all variances in its comparison image.

[0149] 2) If the degree of change value is greater than 1, it means that the change in the degree of change of the grayscale values ​​of each window in the image is greater than the change in the degree of change of the grayscale values ​​of each window in the comparison image, then it is updated to the degree of change value -1, that is, the updated degree of change value reflects the change ratio between the two, that is, the difference between the standard deviation of all variances in the image and the standard deviation of all variances in its comparison image / the standard deviation of all variances in its comparison image.

[0150] 3) If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image. Otherwise, it is determined to be a cut image. Alternatively, if the quotient of the current average change value and the current change degree is not less than the preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

[0151] The change threshold is an empirical value, which is a value between 0 and 1. The change threshold is determined according to the transmission rate and accuracy requirements of the monitoring image. If fast transmission is required and very accurate monitoring images are not required, the change threshold can be slightly larger, so that the probability that the current average change value is not less than the preset change threshold becomes smaller, the possibility of determining it as a retained image also becomes smaller, and the possibility of being cut off becomes larger. If the transmission rate is not limited, but accurate monitoring images are required, the change threshold can be slightly smaller, so that the probability that the current average change value is not less than the preset change threshold becomes larger, the possibility of determining it as a retained image also becomes larger, and the possibility of being cut off becomes smaller.

[0152] The degree threshold is an empirical value, which is a value between 0 and 1. The degree threshold is determined according to the transmission rate and accuracy requirements of the monitoring image. If fast transmission is required and very accurate monitoring images are not required, the degree threshold can be slightly larger, so that the probability that the current change degree value is not less than the preset degree threshold becomes smaller, the possibility of determining it as a retained image also becomes smaller, and the possibility of being cut off becomes larger. If the transmission rate is not limited, but accurate monitoring images are required, the cutting threshold can be slightly smaller, so that the probability that the current change degree value is not less than the preset degree threshold becomes larger, the possibility of being determined as a retained image also becomes larger, and the possibility of being cut off becomes smaller.

[0153] Since the current average change value is a value not greater than 1, it is comparable to the degree threshold.

[0154] If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image. Otherwise, it is determined to be a cut image. This means that compared with the comparison image, the average change value and the change degree value of the image are larger. At this time, the image is considered to be a non-redundant image and is determined to be a retained image.

[0155] If the quotient of the current average change value and the current degree of change is not less than the preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image, which means that compared with the comparison image, the change in the average change value of the image is greater than the change degree value, which means that although the change between each pixel is not large, the overall grayscale value of the image changes more. It is possible that the object has not changed but the color has changed. However, during the continuous acquisition of the same camera device, since the acquisition parameters are unchanged, the color change needs to be paid attention to. At this time, the image is considered to be a non-redundant image and is determined to be a retained image.

[0156] 104-3, cutting out the cut image from the surveillance video.

[0157] Through step 104 - 3 , the cut image can be cut out from the surveillance video, that is, the redundant image is cut out.

[0158] 105, when the transmission trigger is obtained, the cut surveillance video is transmitted.

[0159] For example, after receiving an instruction for triggering transmission sent by an IoT forwarding node (such as a drone), the IoT terminal believes that a transmission trigger has been obtained, and will send all stored cut surveillance videos to the IoT forwarding node, and then delete all stored cut surveillance videos, so that the IoT terminal clears its storage space in order to continue to obtain surveillance videos transmitted by the various field monitoring devices connected to it after they are subsequently turned on.

[0160] This embodiment relates to a method for transmitting field monitoring images, which is executed by an Internet of Things terminal, and includes: obtaining a monitoring video transmitted by any field monitoring device after any startup, wherein the monitoring video is composed of multiple monitoring images; determining a basic offset of the monitoring video; determining a shearing value of each image according to the basic offset; shearing the monitoring video according to the shearing value; and transmitting the sheared monitoring video after obtaining a transmission trigger. Duplicate images can be effectively sheared off, the amount of data in the sheared monitoring video can be reduced, and waste of transmission resources and storage resources can be avoided.

[0161] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0162] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0163] Finally, it should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for transmitting field monitoring images, characterized in that: The method is executed by an Internet of Things terminal, and the method includes: Obtaining a monitoring video transmitted by any field monitoring device after any startup, wherein the monitoring video is composed of a plurality of monitoring images; Determining a basic deviation of the surveillance video; Determining a shearing value of each image according to the basic offset; Cut the monitoring video according to the cutting value; When the transmission trigger is obtained, the cut surveillance video is transmitted.

2. The method according to claim 1, characterized in that The determining of the basic deviation of the surveillance video includes: Acquire the first image of the surveillance video; Acquire a standard image of any field monitoring device; Performing object recognition on the first image to determine a first set of pixels involved in each object; Performing object recognition on the standard image to determine a second set of pixels involved in each object; Determining a first grayscale value of each pixel in the grayscale image of the first image; Determining a second grayscale value of each pixel in the grayscale image of the standard image; For any pixel in the first image, the grayscale value deviation of the any pixel is determined as the absolute value of the difference between the first grayscale value of the any pixel and the second grayscale value of the any pixel; the object quantity deviation of the any pixel is determined as the absolute value of the difference between the first set quantity containing the any pixel and the second set quantity containing the any pixel; the object overall deviation of the any pixel is determined as the absolute value of the difference between the total number of elements in the union of all the first set quantities containing the any pixel and the total number of elements in the union of all the second set quantities containing the any pixel; Determine the offset value of each pixel in the first image according to the gray value deviation of each pixel, the object quantity deviation and the object overall deviation; The basic offset of the surveillance video is determined as the number of pixels in the first image whose offset value is less than a preset offset threshold / the total number of pixels in the first image.

3. The method according to claim 2, characterized in that Determining the offset value of each pixel in the first image according to the gray value deviation of each pixel, the object quantity deviation and the object overall deviation includes: The offset value for any pixel in the first image is e^(grayscale value deviation of any pixel / maximum value of grayscale value deviation of all pixels in the first image + object quantity deviation of any pixel / maximum value of object quantity deviation of all pixels in the first image + overall object deviation of any pixel / maximum value of overall object deviation of all pixels in the first image).

4. The method according to claim 2, characterized in that: The step of determining the shearing value of each image according to the basic offset comprises: Determine a comparison image of each image according to the basic offset; Determine each object in each image and its minimum bounding rectangle; Determine each object and its minimum bounding rectangle in the comparison images of each object; For any image, if the object in the image is the same as the object in the comparison image, determine the overlapping area between the minimum bounding rectangle of each object in the image and the minimum bounding rectangle of the corresponding object in the comparison image; Determining an offset ratio of each object in any one of the images based on the overlapped area; If the maximum value of the offset ratio of objects of all target types in any one of the images is less than a preset offset ratio threshold, the maximum value of the offset ratio of objects of all target types in any one of the images is determined to be the clipping value of any one of the images; otherwise, the clipping value of any one of the images is determined to be 1.

5. The method according to claim 4, characterized in that The determining the offset ratio of each object in any one of the images based on the overlapped area includes: The offset ratio of each object in any one of the images is determined as 1-the ratio of the number of pixels in the overlapping area of ​​the object to the number of pixels in the minimum bounding rectangle of the object in any one of the images.

6. The method according to claim 4, characterized in that The step of determining the comparison image of each image according to the basic deviation comprises: For the first image, if the basic offset is less than the preset offset ratio, and the distribution of pixels in the first image whose offset value is less than the preset offset threshold conforms to the preset distribution, the standard image is determined as its comparison image, otherwise the first image is used as its comparison image; For an image other than the first one, its previous image is determined as its comparison image.

7. The method according to claim 4, characterized in that After determining each object and its minimum bounding rectangle in the comparison image of each object, the method further includes: For any image, if an object in the image is different from an object in a comparison image, determining the type of the different objects; If the type includes the target type, determining the shear value of any one of the images to be 1; If the type does not include the target type, the shearing value of any one of the images is determined to be the number of pixels in the minimum circumscribed rectangle of different objects / the total number of pixels in any one of the images.

8. The method according to claim 1, characterized in that The step of cutting the surveillance video according to the cutting value includes: For each image, if its clipping value is not less than a preset clipping threshold, it is determined to be a retained image; If the shearing value is less than the preset shearing threshold, a square window is constructed with a preset width, starting from the upper left corner of each image and the comparison image of each image, the upper left corner pixel is used as the upper left corner position of the square window, and the square window is moved from left to right and from top to bottom with a step length of 1 pixel, and the variance of the grayscale values ​​of all pixels in the square window is determined each time; the ratio of the mean of all variances in each image to the mean of all variances in its comparison image is determined as the average change value of the image; the ratio of the standard deviation of all variances in the image to the standard deviation of all variances in its comparison image is determined as the change degree value of the image; and whether it is a retained image is determined according to the average change value and the change degree value; The cut image is cut out from the surveillance video.

9. The method according to claim 8, characterized in that The step of determining whether the image is a retained image according to the average change value and the change degree value includes: If the average change value is greater than 1, update it to the average change value - 1; If the change degree value is greater than 1, it is updated to the change degree value - 1; If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

10. The method according to claim 8, characterized in that The step of determining whether the image is a retained image according to the average change value and the change degree value includes: If the average change value is greater than 1, update it to the average change value - 1; If the change degree value is greater than 1, it is updated to the change degree value - 1; If the quotient of the current average change value and the current change degree is not less than the preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

Images