A method for transmitting field monitoring images

Abstract

The present invention relates to a method for transmitting field surveillance images, which is executed by an Internet of Things terminal. The method comprises: obtaining a surveillance video transmitted by any field surveillance device after any power-on, wherein the surveillance video is composed of multiple surveillance images; determining a basic offset of the surveillance video; determining a shearing value of each image based on the basic offset; shearing the surveillance video based on the shearing value; and transmitting the sheared surveillance video after a transmission trigger is obtained. The method of the present invention determines a basic offset of the surveillance video; determines a shearing value of each image based on the basic offset; shearing the surveillance video based on the shearing value; and transmitting the sheared surveillance video after a transmission trigger is obtained. The method can effectively shear out duplicate images, reduce the amount of data in the sheared surveillance video, and avoid wasting transmission and storage resources.

Description

Technical Field

[0001] The present invention relates to the technical field of image transmission, and in particular to a field monitoring image transmission method. Background Art

[0002] As human beings continue to destroy the natural environment, the population of wild animals has decreased sharply. Through field monitoring, we can obtain original data such as the survival of wild animals, which is helpful for the analysis and protection of wild animal populations.

[0003] Currently, field monitoring equipment (such as cameras or infrared cameras) will automatically recognize power-on triggers. When the power-on trigger is recognized, it will automatically start to obtain monitoring video and transmit the monitoring video to the IoT terminal. After the IoT terminal summarizes the monitoring video, it will send it to the IoT forwarding node (such as a drone), and then the intermediate node will transmit it back to the IoT service node (such as a server).

[0004] In actual practice, field monitoring equipment may have problems such as power-on trigger misidentification, which will result in duplicate images in the collected video. These images will not only increase the data volume of the monitoring video and waste transmission resources, but also consume unnecessary storage resources. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] In order to solve the above problems, the present invention provides a field monitoring image transmission method.

[0007] (2) Technical solution

[0008] In order to achieve the above objectives, the main technical solutions adopted by the present invention include:

[0009] A method for transmitting field monitoring images, the method being executed by an Internet of Things terminal, the method comprising:

[0010] Obtain the surveillance video transmitted by any field surveillance device after any startup, where the surveillance video consists of multiple surveillance images;

[0011] Determine the basic deviation of the surveillance video;

[0012] Determine the shear value of each image according to the basic offset;

[0013] Cut the surveillance video according to the cutting value;

[0014] When the transmission trigger is obtained, the cut surveillance video is transmitted.

[0015] Optionally, determining a basic deviation of the surveillance video includes:

[0016] Get the first image of the surveillance video;

[0017] Obtain standard images from any field monitoring equipment;

[0018] Performing object recognition on the first image to determine a first set of pixels associated with each object;

[0019] performing object recognition on the standard image to determine a second set of pixels involved in each object;

[0020] determining a first grayscale value of each pixel in the grayscale image of the first image;

[0021] determining a second grayscale value of each pixel in the grayscale image of the standard image;

[0022] For any pixel in the first image, determine the grayscale value deviation of any pixel as the absolute value of the difference between the first grayscale value of any pixel and the second grayscale value of any pixel; determine the object quantity deviation of any pixel as the absolute value of the difference between the quantity of a first set including any pixel and the quantity of a second set including any pixel; determine the object overall deviation of any pixel as the absolute value of the difference between the total number of elements in the union of all the quantities of the first set including any pixel and the total number of elements in the union of all the quantities of the second set including any pixel;

[0023] Determine the offset value of each pixel in the first image according to the gray value deviation of each pixel, the number deviation of objects and the overall deviation of objects;

[0024] The basic offset of the surveillance video is determined as the number of pixels in the first image whose offset value is less than the preset offset threshold / the total number of pixels in the first image.

[0025] Optionally, determining an offset value of each pixel in the first image according to a grayscale value deviation of each pixel, an object quantity deviation, and an object overall deviation includes:

[0026] The offset value for any pixel in the first image is e^(grayscale value deviation of any pixel / maximum value of grayscale value deviation of all pixels in the first image + object quantity deviation of any pixel / maximum value of object quantity deviation of all pixels in the first image + overall object deviation of any pixel / maximum value of overall object deviation of all pixels in the first image).

[0027] Optionally, determining a shearing value for each image according to a basic offset includes:

[0028] Determine the comparison image of each image according to the basic offset;

[0029] Determine each object in each image and its minimum bounding rectangle;

[0030] Determine each object and its minimum bounding rectangle in the comparison image of each object;

[0031] For any image, if the object in any image is the same as the object in its comparison image, then determine the overlapping area between the minimum bounding rectangle of each object in any image and the minimum bounding rectangle of the corresponding object in its comparison image;

[0032] Determine the offset ratio of each object in any image based on the overlapping area;

[0033] If the maximum value of the offset ratio of objects of all target types in any image is less than a preset offset ratio threshold, the maximum value of the offset ratio of objects of all target types in any image is determined to be the shearing value of any image; otherwise, the shearing value of any image is determined to be 1.

[0034] Optionally, determining an offset ratio of each object in any one of the images based on the overlapping area includes:

[0035] The offset ratio of each object in any image is determined as 1-the ratio of the number of pixels in the overlapped area of ​​the object to the number of pixels in the minimum bounding rectangle of the object in any image.

[0036] Optionally, determining a comparison image for each image according to the basic offset includes:

[0037] For the first image, if the basic offset is less than the preset offset ratio, and the distribution of pixels in the first image whose offset value is less than the preset offset threshold conforms to the preset distribution, the standard image is determined as its comparison image; otherwise, the first image is used as its comparison image;

[0038] For an image that is not the first image, the previous image is determined as the comparison image.

[0039] Optionally, after determining each object and its minimum bounding rectangle in the comparison image of each object, the method further includes:

[0040] For any image, if the object in any image is different from the object in its comparison image, the type of the different object is determined;

[0041] If the type includes the target type, the shear value of any image is determined to be 1;

[0042] If the type does not include the target type, the shear value of any image is determined as the number of pixels in the minimum bounding rectangle of different objects / the total number of pixels in any image.

[0043] Optionally, the surveillance video is cropped according to the crop value, including:

[0044] For each image, if its clipping value is not less than a preset clipping threshold, it is determined to be a retained image;

[0045] If the shearing value is less than a preset shearing threshold, a square window is constructed with a preset width. Starting from the upper left corner of each image and its comparison image, the upper left corner pixel is used as the upper left corner position of the square window. The square window is moved from left to right and from top to bottom in a step length of 1 pixel. The variance of the grayscale values ​​of all pixels in the square window is determined each time. The ratio of the mean of all variances in each image to the mean of all variances in its comparison image is determined as the average change value of the image. The ratio of the standard deviation of all variances in the image to the standard deviation of all variances in its comparison image is determined as the change degree value of the image. Whether it is a retained image is determined based on the average change value and the change degree value.

[0046] Cut out the cut image from the surveillance video.

[0047] Optionally, determining whether it is a retained image according to the average change value and the change degree value includes:

[0048] If the average change value is greater than 1, it will be updated to the average change value - 1;

[0049] If the degree of change value is greater than 1, it is updated to the degree of change value - 1;

[0050] If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

[0051] Optionally, determining whether it is a retained image according to the average change value and the change degree value includes:

[0052] If the average change value is greater than 1, it will be updated to the average change value - 1;

[0053] If the degree of change value is greater than 1, it is updated to the degree of change value - 1;

[0054] If the quotient of the current average change value and the current change degree is not less than a preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cropped image.

[0055] (3) Beneficial effects

[0056] The present invention relates to a method for transmitting field surveillance images, which is executed by an Internet of Things terminal. The method comprises: obtaining a surveillance video transmitted by any field surveillance device after any power-on, wherein the surveillance video is composed of multiple surveillance images; determining a basic offset of the surveillance video; determining a shearing value of each image based on the basic offset; shearing the surveillance video based on the shearing value; and transmitting the sheared surveillance video after a transmission trigger is obtained. The method of the present invention determines a basic offset of the surveillance video; determines a shearing value of each image based on the basic offset; shearing the surveillance video based on the shearing value; and transmitting the sheared surveillance video after a transmission trigger is obtained. The method can effectively shear out duplicate images, reduce the amount of data in the sheared surveillance video, and avoid wasting transmission and storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of a field monitoring image transmission method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0059] Currently, field surveillance equipment (such as video cameras or infrared cameras) automatically recognizes power-on triggers. Once this trigger is detected, it automatically powers on to capture surveillance video and transmits it to an IoT terminal. The IoT terminal aggregates the surveillance video and sends it to an IoT forwarding node (such as a drone), which then transmits it back to an IoT service node (such as a server). In practice, field surveillance equipment can misidentify power-on triggers, resulting in duplicate images in the captured video. These duplicate images not only increase the amount of surveillance video data, waste transmission resources, but also consume unnecessary storage resources.

[0060] To this end, the present invention relates to a method for transmitting field surveillance images, which is executed by an Internet of Things terminal. The method comprises: obtaining a surveillance video transmitted by any field surveillance device after any power-on, wherein the surveillance video is composed of multiple surveillance images; determining a basic offset of the surveillance video; determining a shearing value of each image based on the basic offset; shearing the surveillance video based on the shearing value; and transmitting the sheared surveillance video after a transmission trigger is obtained. The method of the present invention determines a basic offset of the surveillance video; determines a shearing value of each image based on the basic offset; shearing the surveillance video based on the shearing value; and transmitting the sheared surveillance video after a transmission trigger is obtained. The method can effectively shear out duplicate images, reduce the amount of data in the sheared surveillance video, and avoid wasting transmission and storage resources.

[0061] This method is applied to IoT terminals.

[0062] During field monitoring, one or more field monitoring devices (such as video cameras and infrared cameras) can be installed in the monitoring area (e.g., areas frequented by wild animals). Each field monitoring device connects to an IoT terminal using its own pre-configured wireless communication module (e.g., a 2.4 GHz wireless communication module). The field monitoring device and IoT terminal form a local area network connection.

[0063] Each field monitoring device is equipped with an infrared pyroelectric sensor and a camera (such as a camera). When a wild animal is near the field monitoring device (such as a wild animal is 10 meters away from the field monitoring device), its infrared pyroelectric sensor will sense the wild animal, triggering the camera device to turn on and collect monitoring images. The collected multiple monitoring images will be automatically transmitted to the Internet of Things terminal. After a period of time (such as 15 minutes) after the camera device is turned on, or if the infrared pyroelectric sensor does not sense wild animals for a period of time (such as 5 minutes, that is, the infrared pyroelectric sensor does not sense wild animals within 5 minutes), the camera device in the field monitoring device will automatically shut down, thereby stopping the transmission of images to the Internet of Things terminal. Here, all images transmitted to the Internet of Things terminal from power on to power off constitute the monitoring video transmitted by the field monitoring device after the power is turned on.

[0064] Based on data-collecting instructions, IoT forwarding nodes (such as drones) fly to the monitoring area. These nodes establish data transmission links with IoT terminals in the monitoring area and then send instructions to trigger transmission. Upon receiving the transmission trigger, the IoT terminals transmit the surveillance video they receive from the field monitoring devices to the IoT forwarding nodes. These videos are then deleted, allowing the IoT terminals to clear their storage space and continue to receive surveillance video from connected field monitoring devices after they are powered on.

[0065] After the IoT forwarding node obtains the monitoring image transmitted by the IoT terminal, it will transmit the monitoring image to the server for subsequent analysis.

[0066] The method provided in this embodiment is executed by an IoT terminal. Upon acquiring surveillance video transmitted by any field surveillance device after any power-up, the terminal optimizes the video and removes duplicate images, thereby generating a cropped surveillance video. Because the cropped surveillance video contains no duplicate images, the data volume of the cropped surveillance video is significantly reduced compared to the pre-cropped surveillance video. Upon receiving a transmission trigger, the cropped surveillance video is transmitted to an IoT forwarding node, reducing the demand for transmission and storage resources and avoiding waste of these resources.

[0067] See also Figure 1This embodiment provides a method for transmitting field monitoring images. The method is executed by an Internet of Things terminal. The implementation process of the method is as follows:

[0068] 101, obtaining the surveillance video transmitted by any field surveillance device after any startup.

[0069] Among them, the surveillance video is composed of multiple surveillance images.

[0070] The camera in any field surveillance device will activate and capture images when a wild animal approaches. The captured images (i.e., surveillance images) are automatically transmitted to the IoT terminal. This image acquisition and transmission continues until the field surveillance device turns off its camera. All surveillance images captured from the time the camera is turned on to the time it is turned off are automatically transmitted to the IoT terminal. The IoT terminal will compile all captured images into a surveillance video and temporarily store it. Once the surveillance video is cut or the cut video is transmitted, the current surveillance video will be deleted.

[0071] The Internet of Things terminal will cut it using the method provided in this embodiment to ensure that the cut video does not contain repeated surveillance images, and further ensure that the cut video does not contain redundant video.

[0072] The IoT terminal will subsequently store the cropped surveillance video. The cropped surveillance video is deleted only after transmission. Therefore, the cropped surveillance video stored on the IoT terminal is the surveillance video that has not been transmitted back to the server through the IoT forwarding node. The next time a transmission trigger is received, all stored cropped surveillance video will be transmitted. After the transmission, all stored cropped surveillance video will be deleted.

[0073] In step 101, the IoT terminal retrieves the stored surveillance video transmitted by any field surveillance device after any power-on. This surveillance video is the original, uncut video, and therefore contains duplicate surveillance images, indicating image redundancy. The surveillance video acquired in step 101 is subsequently clipped through steps 102 to 104 to remove duplicate surveillance images (i.e., redundant images). This ensures that the clipped surveillance video contains no redundant images, reduces the data volume of the clipped surveillance video, and avoids wasting transmission and storage resources.

[0074] 102. Determine the basic offset of the surveillance video.

[0075] The deviation represents the difference between the surveillance video and the standard image captured by the field surveillance equipment.

[0076] After the field monitoring equipment is installed, a standard image is collected periodically (e.g., daily). This standard image is a normal (wildlife-free) image captured by the field monitoring equipment (e.g., a monitoring image is collected at 8:00 AM daily. This monitoring image shows the normal image captured by the field monitoring equipment that day, including the vegetation included and the basic color tone of the monitoring image caused by weather). The standard image can be used to determine the degree of change in the currently captured monitoring image, which is known as the basic deviation.

[0077] The implementation process of step 102 is as follows:

[0078] 102-1, obtain the first image of the surveillance video.

[0079] The first image is the first image transmitted by any field monitoring device after it is turned on.

[0080] 102-2, obtain a standard image of any field monitoring equipment.

[0081] For example, the standard image is the standard image collected and transmitted by the field monitoring equipment at the last 8 o'clock.

[0082] 102-3, perform object recognition on the first image to determine a first set of pixels involved in each object.

[0083] This step uses the existing object recognition scheme in the image to perform object recognition on the first image, and obtains each object and the pixel identification covered by each object. The identification forms a set, namely the first set. It should be noted that the object here is a complete object, such as a panda eating bamboo leaves. For the bamboo leaves at its mouth, half are in the monitoring image and half are not (eaten into the panda's mouth). After identifying half of the bamboo leaves in the monitoring image, it will be completed by the existing AI scheme. The completed object (including the uneaten part of the bamboo leaves and the part that has been eaten into the mouth) is the object described in this embodiment. Therefore, the object described in this embodiment must be a complete object. In this way, there is a situation where one pixel corresponds to multiple objects, such as the pixel of the panda's mouth, which corresponds to both the panda and the leaves.

[0084] If there are multiple objects in the first image (such as multiple leaves, multiple stones, multiple flowers), etc., each leaf, each stone, each flower, and the pixels covered by each leaf in the first image, the pixels covered by each stone in the first image, and the pixels covered by each flower in the first image will be identified in step 102-3. The pixels covered by each leaf in the first image form a first set of the leaf, the pixels covered by each stone in the first image form a first set of the stone, and the pixels covered by each flower in the first image form a first set of the flower. In this way, there will be as many first sets as there are objects identified in step 102-3. Each first set corresponds to an object. At the same time, the object's identifier will also be identified (such as determining the type of the object, such as a leaf, etc., through an existing object recognition scheme in an image). The type is the identifier of the object, and the identifier is used as an attribute of the first set.

[0085] 102-4, performing object recognition on the standard image to determine a second set of pixels involved in each object.

[0086] The implementation process of this step is similar to step 102-3. Please refer to the description of step 102-3 for details, which will not be elaborated here.

[0087] 102-5, determine a first grayscale value of each pixel in the grayscale image of the first image.

[0088] This step may adopt an existing grayscale image conversion solution to convert the first image into a grayscale image, and then obtain a first grayscale value of each pixel in the grayscale image of the first image.

[0089] 102-6, determine a second grayscale value of each pixel in the grayscale image of the standard image.

[0090] The implementation process of this step is similar to step 102-5. Please refer to the description of step 102-5 for details, which will not be elaborated here.

[0091] 102-7. For any pixel in the first image, determine the grayscale value deviation of the pixel as the absolute value of the difference between the first grayscale value of the pixel and the second grayscale value of the pixel. Determine the object quantity deviation of the pixel as the absolute value of the difference between the quantity of the first set that includes the pixel and the quantity of the second set that includes the pixel. Determine the object overall deviation of the pixel as the absolute value of the difference between the total number of elements in the union of all quantities of the first set that includes the pixel and the total number of elements in the union of all quantities of the second set that includes the pixel.

[0092] For example, for any pixel i, the first grayscale value of pixel i in the first image is G1(i), and the second grayscale value of pixel i in the standard image is G2(i). Then, the grayscale value deviation of pixel i is GD(i) = |G1(i) - G2(i)|. The grayscale value deviation of any pixel represents the difference in grayscale value between the first image and the standard image for the same pixel. A larger value indicates a greater difference in the pixel, indicating that the captured surveillance video is less typical of the environment.

[0093] For any pixel i, determine the number O1(i) of first sets containing pixel i in all first sets, and determine the number O2(i) of second sets containing pixel i in all second sets. Then determine the object quantity deviation OD(i) = |O1(i)-O2(i)| for pixel i. The object quantity deviation of any pixel represents the difference in the number of objects that belong to the same pixel. The greater the difference, the more different the object represented by the pixel (for example, pixel i in the first image is a pixel covered by leaves, but pixel i in the standard image has no object, and at this time, pixel i has a difference in object). The larger the value, the greater the difference of the pixel, and the less the surveillance video collected this time is the normal image of the environment.

[0094] For any pixel i, in all first sets, perform a union operation on all elements in the first set containing pixel i (for example, a panda is eating bamboo leaves, and the panda and bamboo leaves overlap on pixel i, so the objects corresponding to pixel i are leaves and pandas, so the first set of leaves and the first set of pandas will be merged), and obtain the number of elements E1(i) in the set after the union operation. Then E1(i) represents the range of all objects corresponding to pixel i. In all second sets, perform a union operation on all elements in the second set containing pixel i, and obtain the number of elements E2(i) in the set after the union operation. Then determine the overall object deviation ED(i) = |E1(i)-E2(i)| for pixel i. The overall object deviation of any pixel represents the difference in the range of the object corresponding to the same pixel point. The larger the value, the greater the change in the corresponding object, which means that the surveillance video collected this time is less the usual image of the environment.

[0095] 102-8, determining an offset value of each pixel in the first image based on the grayscale value deviation of each pixel, the object quantity deviation, and the object overall deviation.

[0096] The offset value for any pixel in the first image is e^(grayscale value deviation of any pixel / maximum value of grayscale value deviation of all pixels in the first image + object quantity deviation of any pixel / maximum value of object quantity deviation of all pixels in the first image + overall object deviation of any pixel / maximum value of overall object deviation of all pixels in the first image).

[0097] For example, for any pixel i, the offset value D(i) of pixel i = e^(GD(i) / GD(i-max)+OD(i) / OD(i-max)+ED(i) / ED(i-max)).

[0098] Among them, GD(i-max) is the maximum gray value deviation of all pixels in the first image, OD(i-max) is the maximum object quantity deviation of all pixels in the first image, and ED(i-max) is the maximum object overall deviation of all pixels in the first image.

[0099] GD(i) / GD(i-max) represents the grayscale value deviation ratio of pixel i in the first image, OD(i) / OD(i-max) represents the object number deviation ratio of pixel i in the first image, and ED(i) / ED(i-max) represents the overall object deviation ratio of pixel i in the first image. The offset value of any pixel describes the difference between the pixel and the standard image based on three dimensions: the pixel's grayscale value, the corresponding object, and the range of the corresponding object. The smaller the value, the smaller the difference, and the more the collected surveillance video is a normal image of the environment. The larger the value, the greater the difference, and the less the collected surveillance video is a normal image of the environment.

[0100] 102-9, determining the basic offset of the surveillance video as the number of pixels in the first image whose offset value is less than a preset offset threshold / the total number of pixels in the first image.

[0101] The offset threshold is an empirical value, ranging from 0 to 1. It is determined based on the transmission rate and accuracy requirements for surveillance images. If fast transmission is required and highly accurate surveillance images are not required, the offset threshold can be slightly larger. This increases the number of pixels in the first image with offset values ​​less than the preset offset threshold, thus increasing the base offset. The smaller the difference between the first image and the standard image, the closer the image is to the standard image. If the transmission rate is not restricted but accurate surveillance images are required, the offset threshold can be slightly smaller. This decreases the number of pixels in the first image with offset values ​​less than the preset offset threshold, thus reducing the base offset. The larger the difference between the first image and the standard image, the closer the image is to the standard image, the closer the image is to the standard image.

[0102] If the offset value is less than the preset offset threshold, the pixel has changed but the change is negligible and considered unchanged. Otherwise, the pixel is considered changed. Therefore, the basic offset of the surveillance video is the ratio of unchanged pixels between the first image and the standard image. A larger value indicates that the change between the first image and the standard image is smaller, and the more similar they are.

[0103] 103 , determining a shearing value of each image according to the basic offset.

[0104] The shear value represents the degree of repetition of each image, which is the degree of repetition compared to a standard. Therefore, the implementation process of this step is:

[0105] 103-1, determining a comparison image for each image according to the basic offset.

[0106] The comparison image of any image is the standard of that image.

[0107] For the first image, if the base offset is less than the preset offset ratio, and the distribution of pixels in the first image with offset values ​​less than the preset offset threshold conforms to the preset distribution, the comparison image is determined as the base image. Otherwise, the first image is used as the base image. For non-first images, the previous image is used as the base image.

[0108] Among them, the pixel distribution can be obtained based on an existing distribution calculation method, such as determining the pixel distribution through a K function. In addition, a preset distribution such as a uniform distribution is provided. The offset ratio is also a preset empirical value, ranging from 0 to 1, which is determined based on the accuracy requirements of the monitoring image. If a very precise monitoring image is not required, the offset ratio can be slightly larger, and the possibility of the basic offset being less than the preset offset ratio increases, and the possibility of the first image being considered similar to the comparison image increases, and the case of determining the comparison image as its base image increases. If a precise monitoring image is required, the offset ratio can be slightly smaller, and the possibility of the basic offset being less than the preset offset ratio decreases, and the possibility of the first image being considered similar to the comparison image decreases, and the case of determining the first image as its base image increases.

[0109] For the first image, if the basic offset is less than the preset offset ratio, it means that the first image is very similar to the standard image; if the distribution of pixels in the first image with offset values ​​less than the preset offset threshold conforms to the preset uniform distribution, it means that the distribution of identical (not necessarily identical, as long as it is less than the preset offset threshold) pixels in the first image and the standard image is relatively uniform, then it can be considered that the distribution of different pixels (that is, not less than the preset offset threshold) is also uniform. If wild animals appear, the distribution of different pixels must be non-uniform. Therefore, if the basic offset is less than the preset offset ratio and the distribution of pixels in the first image with offset values ​​less than the preset offset threshold conforms to the preset distribution, then the first image is considered to be very similar to, or even identical to, the standard image. At this time, the standard image can be determined as the base image of the first image. Otherwise, it is considered that the first image and the standard image are very different, and comparison cannot be performed according to the standard image. Therefore, the first image will be used as the base image of the first image itself, that is, it is its own base image.

[0110] For non-first images, it is necessary to determine whether they are the same as the previous image, so the previous image is determined as its base image.

[0111] In this way, we only need to compare the difference between each image and the base image to determine whether the image is a redundant image and whether it needs to be cut out.

[0112] 103-2, determine each object in each image and its minimum bounding rectangle.

[0113] The object determination process in this step is similar to step 102-3. For details, see the description of step 102-3 and will not be elaborated here. After obtaining the object, the minimum rectangle containing the object is determined as the minimum bounding rectangle. This process is also the basic process of object recognition and can be achieved using existing object recognition related implementation solutions.

[0114] At the same time, the identity of the object is also identified.

[0115] 103-3, determining each object and its minimum bounding rectangle in the comparison image of each object.

[0116] The implementation process of this step is similar to step 103-2. Please refer to the description of step 103-2 for details, and will not be elaborated here.

[0117] Furthermore, since this is not the first image, its comparison image is the previous image. The objects and their minimum bounding rectangles in this image have already been determined in step 103-2, so the result of step 103-2 can be directly obtained here. For the first image, if its comparison image is itself, then the objects and their minimum bounding rectangles in this image have already been determined in step 103-2, so the result of step 103-2 can be directly obtained here. If its comparison image is a standard image, then the objects in this image have already been identified in step 102-4, so the result of step 102-4 can be directly obtained here. The minimum bounding rectangle of each object can be determined using existing methods in step 103-3.

[0118] At the same time, the identity of the object is also determined.

[0119] 103-4, for any image, if the object in any image is the same as the object in its comparison image, then execute steps 201 to 203. If the object in any image is different from the object in its comparison image, then execute steps 301 and 302.

[0120] Step 103-4 can determine whether the object in any image is the same as the object in its comparison image based on the identifier. If the identifier is the same, the object in any image is considered to be the same as the object in its comparison image; otherwise, the object in any image is considered to be different from the object in its comparison image.

[0121] 201 : Determine an overlapping area between the minimum bounding rectangle of each object in any image and the minimum bounding rectangle of the corresponding object in the comparison image.

[0122] The overlap area represents the change in the area of ​​the same object in two consecutive surveillance images. The smaller the overlap area, the greater the movement of the object, which means the greater the change.

[0123] 202 : Determine an offset ratio of each object in any one image based on the overlapping area.

[0124] The offset ratio of each object in any image is determined as 1-the ratio of the number of pixels in the overlapped area of ​​the object to the number of pixels in the minimum bounding rectangle of the object in any image.

[0125] For any image j, the number of pixels in the minimum bounding rectangle of any object u(j) in image j is NP(u(j)). The number of pixels in the overlapping area of ​​the minimum bounding rectangle of object u(j) in image j and the corresponding minimum bounding rectangle of object u(j) in the comparison image is CNP(u(j)). Then the offset ratio of object u(j) in image j is OR(u(j)) = 1-CNP(u(j)) / NP(u(j)).

[0126] The ratio of the number of pixels in the overlapping area of ​​the object to the number of pixels in the minimum circumscribed rectangle of the object in any image represents the proportion of the overlapping area of ​​the object. The larger the value, the more similar the two are and the smaller the change.

[0127] The offset ratio is the ratio of 1 to the number of pixels in the object's overlapping region, divided by the number of pixels in the object's smallest bounding rectangle in either image. A smaller offset ratio indicates greater similarity and less variation. Therefore, the offset ratio represents the variation of an object between two adjacent images; larger values ​​indicate greater variation.

[0128] 203. If the maximum offset ratio of all target objects of the same type in any image is less than a preset offset ratio threshold, the maximum offset ratio of all target objects of the same type in any image is determined as a cropping value for the image. Otherwise, the cropping value for the image is determined to be 1.

[0129] The offset ratio threshold is an empirical value, ranging from 0 to 1. It is determined based on the transmission rate and accuracy requirements for surveillance images. If fast transmission is required and highly accurate surveillance images are not required, the offset ratio threshold can be slightly larger. This increases the likelihood that the maximum offset ratio of all target types will be less than the preset offset ratio threshold, reducing the number of images with a clipping value of 1 and subsequently increasing the number of surveillance images clipped. If the transmission rate is not restricted but accurate surveillance images are required, the offset ratio threshold can be slightly smaller. This reduces the likelihood that the maximum offset ratio of all target types will be less than the preset offset ratio threshold, increasing the number of images with a clipping value of 1 and subsequently reducing the number of surveillance images clipped.

[0130] In addition, the target type is also a preset type, which is determined based on the monitoring purpose. For example, if monitoring the survival of giant pandas in the wild, the target type is giant pandas. If monitoring the birth of bears in the wild, the target type is bears.

[0131] If the maximum offset ratio of all target types in any image is less than a preset offset ratio threshold, it indicates that the change between the previous and next images for each target type is not significant. The maximum offset ratio of all target types in any image is then determined to be the clipping value for that image. The clipping value represents the change between two adjacent images and ranges from 0 to 1, with larger values ​​indicating greater change. If the maximum offset ratio of at least one target type in any image is not less than a preset offset ratio threshold, it is considered that the change in that object is significant. Therefore, the image is definitely different from the previous image and is not a simple image. Therefore, the clipping value for that image is determined to be 1.

[0132] 301 , for any image, if an object in any image is different from an object in its comparison image, determine the type of the different object.

[0133] This indicates that a new object appears in two consecutive images, or an object disappears, that is, there is a change in the object in any of the images (it may be that there are more objects in any of the images, or there may be less objects in any of the images, or there may be both more and less objects in any of the images).

[0134] 302. If the type includes the target type, it means that the changed object is of interest to the monitoring task. Therefore, the clipping value of each image is determined to be 1. If the type does not include the target type, it means that the changed object is not of interest to the monitoring task. The clipping value of each image is determined to be the number of pixels in the smallest bounding rectangle of different objects / the total number of pixels in each image.

[0135] 104, cutting the surveillance video according to the cutting value.

[0136] The implementation process of step 104 is:

[0137] 104-1, for each image, if its cropping value is not less than a preset cropping threshold, it is determined to be a retained image.

[0138] The cropping threshold is an empirical value, ranging from 0 to 1. It is determined based on the transmission rate and accuracy requirements for surveillance images. If fast transmission is required and precise surveillance images are not required, a larger cropping threshold can be used. This will result in fewer images being retained, leading to more images being cropped. If the transmission rate is not limited but precise surveillance images are required, a smaller cropping threshold can be used. This will result in more images being retained, leading to fewer images being cropped.

[0139] In step 104-2, if the cropping value is less than a preset cropping threshold, a square window with a preset width is constructed. Starting from the upper left corner of each image and its comparison image, the upper left corner pixel is used as the upper left corner position of the square window. The square window is moved from left to right and from top to bottom in steps of one pixel. The variance of the grayscale values ​​of all pixels in the square window is determined each time. The ratio of the mean of all variances in each image to the mean of all variances in its comparison image is determined as the average change value of the image. The ratio of the standard deviation of all variances in the image to the standard deviation of all variances in its comparison image is determined as the change degree value of the image. Based on the average change value and the change degree value, it is determined whether the image is retained.

[0140] That is, if the clipping value is less than the preset clipping threshold, it means that it may be a retained image or not, and the following steps are performed:

[0141] 1. Construct a square window with a preset width (such as a 3×3 square window). Starting from the upper left corner of each image and the comparison image of each image, use the upper left corner pixel as the upper left corner position of the square window, and move the square window from left to right and from top to bottom with a step length of 1 pixel (that is, starting from the upper left corner of the image, move the square window from left to right and from top to bottom, and move one pixel at a time). Determine the variance of the grayscale values ​​of all pixels in the square window each time (the variance represents the change in the grayscale values ​​of all pixels in the square window).

[0142] 2. The ratio of the mean of all variances in each image (representing the mean of the change in grayscale value obtained by changing one pixel in the image) to the mean of all variances in its comparison image (representing the mean of the change in grayscale value obtained by changing one pixel in the comparison image) is determined as the average change value of the image (the ratio represents the difference in the average change in grayscale value between the image and the comparison image, that is, the average change value).

[0143] The average change value represents the difference in pixel values ​​between the image and its comparison image as a whole. This value represents the difference in color between the two images and is a macroscopic change of the image as a whole.

[0144] 3. The ratio of the standard deviation of all variances in the image (characterizing the variation of the grayscale value variation obtained by each pixel change in the image, i.e., the variation of the grayscale value variation of each window) to the standard deviation of all variances in the comparison image (characterizing the variation of the grayscale value variation obtained by each pixel change in the comparison image, i.e., the variation of the grayscale value variation of each window) is determined as the variation value of the image (the ratio is the variation value, characterizing the difference in variation between the grayscale value variation of the image and the comparison image, i.e., the variation of the grayscale value variation of each window in the image compared to the variation of the grayscale value variation of each window in the comparison image).

[0145] The degree of change value represents the degree of change in pixel values ​​between the pixels of an image and its comparison image as a whole. This value represents the change between the pixel values ​​of each pixel point in each window of the two images, and is a microscopic change based on each pixel in the image.

[0146] 4. Determine whether it is a retained image based on the average change value and the change degree value.

[0147] This can be achieved by following the steps below.

[0148] 1) If the average change value is greater than 1, it means that the mean of the grayscale value change of the image is greater than the mean of the grayscale value change of its comparison image. Then it is updated to the average change value - 1. The updated average change value reflects the change ratio between the two, that is, the difference between the mean of all variances in the image and the mean of all variances in its comparison image / the mean of all variances in its comparison image.

[0149] 2) If the degree of change value is greater than 1, it means that the change in the grayscale value change degree of each window in the image is greater than the change in the grayscale value change degree of each window in the comparison image. Then it is updated to the degree of change value -1, that is, the updated degree of change value reflects the change ratio between the two, that is, the difference between the standard deviation of all variances in the image and the standard deviation of all variances in its comparison image / the standard deviation of all variances in its comparison image.

[0150] 3) If the current average change value is not less than a preset change threshold, and the current degree of change value is not less than a preset degree threshold, the image is determined to be a retained image. Otherwise, the image is determined to be a cropped image. Alternatively, if the quotient of the current average change value and the current degree of change is not less than a preset ratio threshold, the image is determined to be a retained image; otherwise, the image is determined to be a cropped image.

[0151] The change threshold is an empirical value, ranging from 0 to 1. It is determined based on the transmission rate and accuracy requirements for surveillance images. If fast transmission is required and highly accurate surveillance images are not required, the change threshold can be slightly higher. This reduces the probability that the current average change value is no less than the preset change threshold, lowers the likelihood of the image being retained, and increases the likelihood of it being clipped. If the transmission rate is not restricted but accurate surveillance images are required, the change threshold can be slightly lower. This increases the probability that the current average change value is no less than the preset change threshold, higher the likelihood of the image being retained, and lowers the likelihood of it being clipped.

[0152] The degree threshold is an empirical value, ranging from 0 to 1. It is determined based on the transmission rate and accuracy requirements for surveillance images. If fast transmission is required and highly accurate surveillance images are not required, the degree threshold can be slightly higher. This reduces the probability that the current degree of change value is no less than the preset degree threshold, lowering the likelihood that the image will be retained and increasing the likelihood that it will be clipped. If the transmission rate is not restricted but accurate surveillance images are required, the clipping threshold can be slightly lower. This increases the probability that the current degree of change value is no less than the preset degree threshold, increasing the likelihood that the image will be retained and decreasing the likelihood that it will be clipped.

[0153] Since the current average change value is a value not greater than 1, it is comparable to the degree threshold.

[0154] If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image. Otherwise, it is determined to be a cut image. This means that compared with the comparison image, the average change value and the change degree value of the image are both larger. At this time, the image is considered to be a non-redundant image and is determined to be a retained image.

[0155] If the quotient of the current average change value and the current degree of change is not less than the preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image, which means that compared with the comparison image, the change in the average change value of the image is greater than the change degree value, which means that although the change between each pixel is not large, the overall grayscale value of the image changes more. It is possible that the object has not changed but the color has changed. However, during the continuous acquisition of the same camera device, since the acquisition parameters are unchanged, the color change needs to be paid attention to. At this time, the image is considered to be a non-redundant image and is determined to be a retained image.

[0156] 104-3, cut out the cut image from the surveillance video.

[0157] Through step 104 - 3 , the cut image can be cut out from the surveillance video, that is, the redundant image is cut out.

[0158] 105. When a transmission trigger is obtained, the cut surveillance video is transmitted.

[0159] If the IoT terminal receives an instruction for triggering transmission sent by an IoT forwarding node (such as a drone), it will believe that the transmission trigger has been obtained, and will send all the stored cut surveillance videos to the IoT forwarding node, and then delete all the stored cut surveillance videos, so that the IoT terminal clears its storage space so that it can continue to obtain the surveillance videos transmitted by the various field monitoring devices connected to it after they are turned on.

[0160] This embodiment relates to a method for transmitting field surveillance images, executed by an Internet of Things (IoT) terminal. The method comprises: obtaining surveillance video transmitted by any field surveillance device after any power-on, the surveillance video consisting of multiple surveillance images; determining a base offset for the surveillance video; determining a cropping value for each image based on the base offset; cropping the surveillance video based on the cropping value; and, upon receiving a transmission trigger, transmitting the cropped surveillance video. This method effectively crops duplicate images, reduces the data volume of the cropped surveillance video, and avoids wasting transmission and storage resources.

[0161] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0162] It should also be noted that the exemplary embodiments described herein describe methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the steps described above. In other words, the steps may be performed in the order described in the embodiments, or in a different order, or several steps may be performed simultaneously.

[0163] Finally, it should be noted that the embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A field monitoring image transmission method, characterized in that: The method is executed by an Internet of Things terminal, and the method includes: Obtaining a surveillance video transmitted by any field surveillance device after any startup, wherein the surveillance video is composed of multiple surveillance images; Determining a basic offset of the surveillance video; Determining a shearing value for each image according to the basic offset; Cutting the surveillance video according to the cutting value; When a transmission trigger is obtained, the cut surveillance video is transmitted; Determining the basic offset of the surveillance video includes: Acquire the first image of the surveillance video; Acquire a standard image of any field monitoring device; Performing object recognition on the first image to determine a first set of pixels related to each object; the first image is the first image transmitted by the monitoring device after any power-on; Performing object recognition on the standard image to determine a second set of pixels related to each object; the standard image is an image without a monitored target captured by the monitoring device; determining a first grayscale value of each pixel in the grayscale image of the first image; determining a second grayscale value of each pixel in the grayscale image of the standard image; For any pixel in the first image, determine the grayscale value deviation of the any pixel as the absolute value of the difference between the first grayscale value of the any pixel and the second grayscale value of the any pixel; determine the object quantity deviation of the any pixel as the absolute value of the difference between the quantity of a first set including the any pixel and the quantity of a second set including the any pixel; determine the object overall deviation of the any pixel as the absolute value of the difference between the total number of elements in the union of all the quantities of the first set including the any pixel and the total number of elements in the union of all the quantities of the second set including the any pixel; Determine the offset value of each pixel in the first image according to the gray value deviation of each pixel, the number deviation of objects and the overall deviation of objects; Determine the basic offset of the surveillance video as the number of pixels in the first image whose offset value is less than a preset offset threshold / the total number of pixels in the first image; Determining the offset value of each pixel in the first image according to the grayscale value deviation of each pixel, the object quantity deviation, and the object overall deviation includes: The offset value for any pixel in the first image is e^(grayscale value deviation of any pixel / maximum grayscale value deviation of all pixels in the first image + object number deviation of any pixel / maximum object number deviation of all pixels in the first image + overall object deviation of any pixel / maximum overall object deviation of all pixels in the first image); The determining the shearing value of each image according to the basic offset includes: Determining a comparison image for each image according to the basic offset; Determine each object in each image and its minimum bounding rectangle; Determine each object and its minimum bounding rectangle in the comparison image of each object; For any image, if the object in the image is the same as the object in the comparison image, determining the overlapping area between the minimum bounding rectangle of each object in the image and the minimum bounding rectangle of the corresponding object in the comparison image; determining an offset ratio of each object in any one of the images based on the overlapping area; If the maximum value of the offset ratios of all target types in any one of the images is less than a preset offset ratio threshold, determining the maximum value of the offset ratios of all target types in any one of the images as the shearing value of the any one of the images; otherwise, determining the shearing value of the any one of the images as 1; The step of cutting the surveillance video according to the cutting value includes: For each image, if its clipping value is not less than a preset clipping threshold, it is determined to be a retained image; If the shearing value is less than a preset shearing threshold, a square window is constructed with a preset width, starting from the upper left corner of each image and the comparison image of each image, with the upper left corner pixel as the upper left corner position of the square window, and the square window is moved from left to right and from top to bottom with a step length of 1 pixel, and the variance of the grayscale values ​​of all pixels in the square window is determined each time; the ratio of the mean of all variances in each image to the mean of all variances in its comparison image is determined as the average change value of the image; the ratio of the standard deviation of all variances in the image to the standard deviation of all variances in its comparison image is determined as the change degree value of the image; and whether it is a retained image is determined based on the average change value and the change degree value; The cut image is cut out from the surveillance video.

2. The method according to claim 1, characterized in that The determining the offset ratio of each object in any one of the images based on the overlapping area includes: The offset ratio of each object in any one of the images is determined as 1-the ratio of the number of pixels in the overlapping area of ​​the object to the number of pixels in the minimum bounding rectangle of the object in any one of the images.

3. The method according to claim 1, characterized in that Determining a comparison image of each image according to the basic offset includes: For the first image, if the basic offset is less than the preset offset ratio, and the distribution of pixels in the first image whose offset value is less than the preset offset threshold conforms to the preset distribution, the standard image is determined as the comparison image; otherwise, the first image is used as the comparison image; For an image that is not the first image, the previous image is determined as the comparison image.

4. The method according to claim 1, wherein After determining each object and its minimum bounding rectangle in the comparison image of each object, the method further includes: For any image, if an object in the image is different from an object in a comparison image, determining the type of the different objects; If the type includes the target type, determining the shear value of any one of the images to be 1; If the type does not include the target type, the shearing value of any one of the images is determined to be the number of pixels in the minimum circumscribed rectangle of different objects / the total number of pixels in any one of the images.

5. The method according to claim 1, wherein The determining whether it is a retained image according to the average change value and the change degree value includes: If the average change value is greater than 1, it will be updated to the average change value - 1; If the degree of change value is greater than 1, it is updated to the degree of change value - 1; If the current average change value is not less than the preset change threshold, and the current change degree value is not less than the preset degree threshold, it is determined to be a retained image; otherwise, it is determined to be a cut image.

6. The method according to claim 1, characterized in that The determining whether it is a retained image according to the average change value and the change degree value includes: If the average change value is greater than 1, it will be updated to the average change value - 1; If the degree of change value is greater than 1, it is updated to the degree of change value - 1; If the quotient of the current average change value and the current change degree is not less than a preset ratio threshold, it is determined to be a retained image; otherwise, it is determined to be a cropped image.

Images