Visual data intelligent collection system based on the Internet of Things

Abstract

The present invention relates to the field of electronic digital data processing technology, and in particular to a visual data intelligent acquisition system based on the Internet of Things, the system comprising a processor and a memory, the processor executing a computer program stored in the memory to implement the following steps: obtaining a suspected ghost pixel set, a non-ghost pixel set and a difference feature mean corresponding to each edge pixel on an edge, obtaining a motion blur degree characterization value corresponding to each frame image according to the suspected ghost pixel set, the non-ghost pixel set and the difference feature mean, obtaining a motion blur degree prediction value of the next frame image according to the motion blur degree characterization value corresponding to each frame image; obtaining a capture frame rate of the next frame image according to the motion blur degree prediction value, and acquiring the next frame image at the capture frame rate of the next frame image. The present invention can improve the quality of the captured image and reduce the generation of redundant visual data.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a visual data intelligent acquisition system based on the Internet of Things. Background Art

[0002] With the continuous development of Internet of Things technology, many fields are currently integrated with Internet of Things technology, such as the field of intelligent visual data collection. The intelligent visual data collection system based on the Internet of Things has a close connection in data collection and processing, remote access and control, data analysis, data sharing and interoperability, and intelligent decision-making, which jointly promotes the development and application of intelligent monitoring. The application fields of the intelligent visual data collection system based on the Internet of Things are also very broad, such as smart cities, traffic management, security monitoring, industrial automation and other fields.

[0003] Since the current intelligent visual data acquisition system based on the Internet of Things usually uses cameras, sensors and other devices to collect visual data, but in the process of collecting visual data in the monitored area, it is generally collected at a fixed acquisition frame rate. However, when collecting based on a fixed acquisition frame rate, not only may redundant visual data be generated, wasting more storage space, but the collected visual data may also be blurred, that is, the quality of the collected visual data is low or the collected image is blurred. If there are no dynamic objects in the monitored area or the collected area, then if the collection is performed at a higher acquisition frame rate, more redundant data will be generated. If there are dynamic objects in the monitored area or the collected area, then if the collection is performed at a lower acquisition frame rate, the collected visual data may be blurred. Therefore, how to adaptively adjust the acquisition frame rate to improve the quality of the collected visual data and reduce the generation of redundant visual data has become an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a visual data intelligent acquisition system based on the Internet of Things, and the technical solutions adopted are as follows:

[0005] An embodiment of the present invention provides a visual data intelligent collection system based on the Internet of Things, comprising the following steps:

[0006] Acquire visual data to be analyzed collected by a target acquisition device, wherein the visual data to be analyzed consists of A frame images, where A is a positive integer greater than 0, and the visual data to be analyzed includes a current frame image collected at a current acquisition time;

[0007] Performing edge detection on each frame image in the visual data to be analyzed to obtain all edges in each frame image, and obtaining a first neighborhood set and a second neighborhood set corresponding to each edge pixel point on the edge according to pixel points on both sides of the edge;

[0008] According to the grayscale values ​​of the pixels in the first neighborhood set and the second neighborhood set, obtaining a suspected ghost pixel set, a non-ghost pixel set, and a difference feature mean corresponding to each edge pixel on the edge;

[0009] According to the suspected ghost pixel set, the non-ghost pixel set and the difference feature mean, a motion blur degree representation value corresponding to each frame image is obtained; according to the motion blur degree representation value corresponding to each frame image, a motion blur degree prediction value of the next frame image is obtained;

[0010] According to the motion blur degree prediction value, the acquisition frame rate of the next frame image is obtained, and the next frame image is acquired by acquiring the next frame image at the acquisition frame rate of the next frame image.

[0011] Beneficial effects: The present invention first obtains the visual data to be analyzed collected by the target acquisition device, and the visual data to be analyzed includes the current frame image collected at the current acquisition time; then, edge detection is performed on each frame image in the visual data to be analyzed to obtain all edges in each frame image, and the first neighborhood set and the second neighborhood set corresponding to each edge pixel point on the edge are obtained according to the pixel points on both sides of the edge, and the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean value corresponding to each edge pixel point on the edge are obtained according to the pixel point grayscale values ​​in the first neighborhood set and the second neighborhood set, and the motion blur degree characterization value corresponding to each frame image is obtained according to the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean value, and the motion blur degree prediction value of the next frame image is obtained according to the motion blur degree characterization value corresponding to each frame image; finally, according to the motion blur degree prediction value, the acquisition frame rate of the next frame image is obtained, and the next frame image is acquired at the acquisition frame rate of the next frame image. The acquisition frame rate of the next frame image obtained by the present invention on the basis of the motion blur degree characterization value corresponding to each frame image can improve the quality of the acquired visual data and reduce the generation of redundant visual data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 This is a flow chart of a method for intelligently collecting visual data based on the Internet of Things of the present invention;

[0014] Figure 2 This is a schematic diagram of ghosting or blurring. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the embodiments of the present invention.

[0016] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0017] This embodiment provides a visual data intelligent collection system based on the Internet of Things, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a visual data intelligent collection method based on the Internet of Things, such as Figure 1 As shown, the method for intelligent collection of visual data based on the Internet of Things includes the following steps:

[0018] Step S001, obtaining visual data to be analyzed collected by a target collection device.

[0019] In the process of collecting visual data of the monitored area, not only redundant visual data may be generated, wasting a lot of storage space, but also blurring or ghosting may occur in the collected visual data, that is, the quality of the collected visual data may be low or the collected visual image may be blurred or ghosted. The blurring or ghosting phenomenon refers to the movement of objects passing through the monitored area. The visual data refers to the image, that is, objects moving at high speed in the collected image may have ghosting phenomenon, and ghosting will make the collected image blurred. If there is no dynamic object in the monitored area or the collected area, then if the collection is performed at a higher collection frame rate, more redundant data will be generated. If there is a dynamic object in the monitored area or the collected area, then if the collection is performed at a lower collection frame rate, the collected visual data may be blurred. For example, Figure 2 is a schematic diagram of ghosting or blurring. Figure 2 The ghosting or blurring phenomenon is due to Figure 2This is caused by the high-speed movement of the CRRC and the inappropriate acquisition frame rate, so the main purpose of this embodiment is to adaptively adjust the acquisition frame rate during the visual data acquisition process, so as to improve the quality of the acquired visual data and reduce the generation of redundant visual data, that is, to improve the quality of the acquired visual images and reduce the generation of redundant images.

[0020] In addition, for ease of understanding, this embodiment will be analyzed by taking any acquisition device as an example. That is, the acquisition devices that appear later in this embodiment are all the same acquisition device and are recorded as target acquisition devices. The acquisition device is fixedly installed at a certain position, and the specific position needs to be set by the implementer according to actual conditions. In this embodiment, the acquisition device can be a camera, a camera or other device. Since the acquisition devices in this embodiment are the same acquisition device, the areas collected or monitored by the target acquisition device in this embodiment are the same, except that during the acquisition process, there will be moving objects passing through the area collected or monitored by the target acquisition device. The moving objects can refer to people or objects, such as various moving vehicles.

[0021] In order to improve the quality of the collected visual data and reduce the generation of redundant visual data, the present embodiment will subsequently predict the degree of motion blur of the next frame image by analyzing the degrees of motion blur of the historical frame image and the current frame image, and determine the acquisition frame rate of the target acquisition device when acquiring the next frame image based on the predicted degree of motion blur of the next frame image, and the degree of motion blur can reflect the degree of blur of the corresponding image caused by the passage of the moving object; therefore, based on the above analysis, it can be known that the present embodiment first needs to obtain the historical frame image and the current frame image, that is, to obtain the visual data to be analyzed collected by the target acquisition device, the visual data to be analyzed is collected and acquired by the target acquisition device at a preset acquisition frame rate, the visual data to be analyzed is composed of A frame images, A is a positive integer greater than 0, and the visual data to be analyzed includes the historical frame image and the current frame image, wherein the last frame image in the visual data to be analyzed is the current frame image, and the current frame image is the image collected by the target acquisition device at the current acquisition moment.

[0022] In addition, this embodiment sets the preset acquisition frame rate to an empirical value, and in general the preset acquisition frame rate is relatively low, such as it can be set to 15FPS; and in this embodiment, the implementer can set the value of A according to actual conditions, such as in this embodiment the value of A can be set to 20 or 10.

[0023] Therefore, this embodiment can obtain the visual data to be analyzed collected by the target acquisition device through the above process; in addition, it should be noted that all images in the visual data to be analyzed are images after grayscale processing.

[0024] Step S002, performing edge detection on each frame image in the visual data to be analyzed, obtaining all edges in the each frame image, and obtaining a first neighborhood set and a second neighborhood set corresponding to each edge pixel point on the edge according to the pixel points on both sides of the edge; obtaining a suspected ghost pixel point set, a non-ghost pixel point set and a difference feature mean corresponding to each edge pixel point on the edge according to the grayscale values ​​of the pixels in the first neighborhood pixel point set and the second neighborhood pixel point set; obtaining a motion blur degree representation value corresponding to each frame image according to the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean; obtaining a motion blur degree prediction for the next frame image according to the motion blur degree representation value corresponding to each frame image.

[0025] Based on the above description, it can be known that the present embodiment achieves the purpose of improving the quality of the collected visual data and reducing the generation of redundant visual data by predicting the degree of motion blur of the next frame of image. If the degree of motion blur of the next frame of image is to be predicted more reliably and accurately, it is necessary to first obtain the degree of motion blur of the image that has been collected, and predict the degree of motion blur of the next frame of image based on the degree of motion blur of the image that has been collected. Therefore, the present embodiment next needs to obtain the motion blur degree representation value corresponding to each frame of image in the visual data to be analyzed. However, in the present embodiment, before obtaining the motion blur degree representation value corresponding to each frame of image, it is necessary to first obtain the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean corresponding to the edge pixel points on each edge of each frame of image, that is, based on the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean are obtained. The motion blur degree characterization value corresponding to each frame image is determined by using a suspected ghost pixel set, a non-ghost pixel set and a difference feature mean. However, before obtaining the suspected ghost pixel set, the non-ghost pixel set and the difference feature mean, it is also necessary to first obtain the first neighborhood set and the second neighborhood set corresponding to the edge pixel points on each edge of each frame image, that is, based on the grayscale values ​​of the pixels in the first neighborhood set and the second neighborhood set, the suspected ghost pixel set, the non-ghost pixel set and the difference feature mean corresponding to the edge pixel points on each edge of each frame image can be obtained. Therefore, based on the above analysis, it can be seen that in this embodiment, it is necessary to first obtain the first neighborhood set and the second neighborhood set corresponding to the edge pixel points on each edge of each frame image. Then the specific process of obtaining the first neighborhood set and the second neighborhood set is:

[0026] First, edge detection is performed on each frame image in the visual data to be analyzed using an edge detection operator to obtain all edges in each frame image in the visual data to be analyzed. In specific applications, the implementer can select an edge detection operator to detect image edges according to actual conditions. For example, in this embodiment, a bidirectional edge detection operator can be used to detect edges in an image.

[0027] Then, according to the pixels on both sides of all edges in each frame image, the first neighborhood set and the second neighborhood set corresponding to each edge pixel point on each edge in each frame image are obtained; and for ease of understanding, this embodiment will be described by taking the process of obtaining the first neighborhood set and the second neighborhood set corresponding to any edge pixel point b on any edge B on any frame image in the visual data to be analyzed as an example, that is, the process of obtaining the first neighborhood set and the second neighborhood set corresponding to the edge pixel point b on the edge B is:

[0028] First, obtain the first neighborhood window and the second neighborhood window corresponding to the edge pixel point b, and then record the set constructed by all the pixels in the first neighborhood window corresponding to the edge pixel point b that do not belong to the edge B as the first neighborhood set corresponding to the edge pixel point b. Similarly, record the set constructed by all the pixels in the second neighborhood window corresponding to the edge pixel point b that do not belong to the edge B as the second neighborhood set corresponding to the edge pixel point b. The pixel points in the neighborhood window that do not belong to the edge B refer to the pixel points located in the corresponding neighborhood window but not on the edge B.

[0029] In this embodiment, the method for obtaining the first neighborhood window and the second neighborhood window corresponding to the edge pixel point b is as follows: first, the tangent of the edge pixel point b located on the edge B is obtained, and recorded as the first tangent; then, a straight line perpendicular to the first tangent is drawn through the edge pixel point b, and recorded as the first vertical line; then, the two pixel points located on the first vertical line and closest to the edge pixel point b are recorded as the first pixel point and the second pixel point, respectively; then, the nearest neighbor pixel point of the first pixel point and the nearest neighbor pixel point of the second pixel point are obtained, and recorded as the first feature point and the second feature point of the edge pixel point b, respectively, and the first feature point and the second feature point of the edge pixel point b are not the edge pixel point b, and the first feature point and the second feature point of the edge pixel point b are both located on the first vertical line; then, two neighborhood windows are constructed with the first feature point and the second feature point as the center points, respectively. The first and second neighborhood windows of the edge pixel point b are recorded as the first and second neighborhood windows respectively. The sides of the neighborhood windows are parallel to the sides of the image. In addition, the value of a needs to be set by the implementer according to the actual situation. For example, in this embodiment, the value of a is set to 3, so the size of the neighborhood window in this embodiment is .

[0030] It should be noted that the first curve corresponding to edge B does not intersect with the second curve corresponding to edge B, and the first curve corresponding to edge B is a curve obtained by connecting the center points of the first neighborhood window corresponding to all edge pixels on edge B, and the second curve corresponding to edge B is a curve obtained by connecting the center points of the second neighborhood window corresponding to all edge pixels on edge B, that is, in this embodiment, the center point of the first neighborhood window corresponding to all edge pixels on edge B is located on one side of edge B, and the center point of the second neighborhood window corresponding to all edge pixels on edge B is located on the other side of edge B, or, the first neighborhood set corresponding to all edge pixels on edge B is located on one side of edge B, and the second neighborhood set corresponding to all edge pixels on edge B is located on the other side of edge B.

[0031] Therefore, based on the above analysis, it can be known that the present embodiment can obtain the first neighborhood set and the second neighborhood set corresponding to each edge pixel point on each edge in each frame image, and it can be known that the first neighborhood set corresponding to the edge pixel point on the same edge is located on the same side of the corresponding edge, the second neighborhood set corresponding to the edge pixel point on the same edge is located on the same side of the corresponding edge, and the first neighborhood set and the second neighborhood set corresponding to the edge pixel point on the same edge are not located on the same side of the corresponding edge, that is, the first neighborhood set and the second neighborhood set corresponding to the edge pixel point on the same edge are respectively located on both sides of the corresponding edge; in addition, the reason why the present embodiment obtains the first neighborhood set and the second neighborhood set located on both sides of the edge is because the image blurring phenomenon caused by the movement of the object or the ghosting phenomenon caused by the moving object in the image, and if the image has the blurring phenomenon caused by the movement of the object, then the two sides of the edge of the moving object will present different characteristics, and the greater the grayscale chaos on which side, the greater the probability of the ghosting area on this side, and the smaller the probability of the ghosting area on the other side, and the ghosting refers to the ghosting of the moving object in the image caused by the movement of the object, such as Figure 2 The red rectangular area in the figure is the ghosting area, so this embodiment can determine the possibility that each edge is a motion ghosting edge by analyzing the grayscale on both sides of the edge. The motion ghosting edge is the edge of the ghosting area of ​​the moving object in the image, and the analysis of the motion ghosting edge can reflect the degree of blurring of the image caused by the movement of the object in the monitored area or the collected area when the corresponding image is collected; in addition, it should be noted that the edges analyzed in this embodiment do not include closed edges, that is, there are no closed edges among all the edges appearing in this embodiment.

[0032] In order to obtain the motion blur degree characterization value corresponding to each frame image, the present embodiment then needs to obtain the suspected ghost pixel set, non-ghost pixel set and difference feature mean corresponding to each edge pixel point on each edge according to the first neighborhood set and the second neighborhood set corresponding to each edge pixel point on each edge in each frame image. The suspected ghost pixel set, non-ghost pixel set and difference feature mean are the key to subsequently obtaining the motion blur degree characterization value corresponding to each frame image; in addition, also for the sake of ease of understanding, the subsequent description will take the suspected ghost pixel set, non-ghost pixel set and difference feature mean corresponding to each edge pixel point on edge B on any frame image in the visual data to be analyzed as an example, that is, the suspected ghost pixel set, non-ghost pixel set and difference feature mean corresponding to each edge pixel point on edge B. The acquisition process is:

[0033] First, the first mean and the second mean corresponding to each edge pixel point on edge B are obtained, and the cumulative sum of the first mean values ​​corresponding to all edge pixel points on edge B is recorded as the first chaotic distribution degree corresponding to edge B, and the cumulative sum of the second mean values ​​corresponding to all edge pixel points on edge B is recorded as the second chaotic distribution degree corresponding to edge B. The first chaotic distribution degree corresponding to edge B represents the chaotic distribution degree of the grayscale distribution of pixels in the area where the first neighborhood set corresponding to all edge pixel points on edge B is located, and the greater the first chaotic distribution degree corresponding to edge B, the more it indicates that the grayscale distribution of pixels in the area where the first neighborhood set corresponding to all edge pixel points on edge B is located is chaotic. The more chaotic, the second chaotic distribution degree corresponding to edge B is characterized by the chaotic grayscale distribution degree of the pixels in the area where the second neighborhood set corresponding to all edge pixels on edge B is located, and the greater the second chaotic distribution degree corresponding to edge B, the more chaotic the grayscale distribution of the pixels in the area where the second neighborhood set corresponding to all edge pixels on edge B is located; then, it is determined whether the first chaotic distribution degree corresponding to edge B is greater than the second chaotic distribution degree corresponding to edge B. If so, it is determined that the area where the first neighborhood set corresponding to the edge pixels on edge B is located is likely to be a ghosting area, and the area where the second neighborhood set corresponding to the edge pixels on edge B is located is determined to be a non-ghosting area. If the possibility of the first neighborhood set corresponding to all edge pixels on edge B is large, then the first neighborhood sets corresponding to all edge pixels on edge B are recorded as the suspected ghost pixel sets corresponding to the corresponding edge pixels, the first means corresponding to each edge pixel on edge B are recorded as the difference feature means corresponding to the corresponding edge pixels, and the second neighborhood sets corresponding to all edge pixels on edge B are recorded as the non-ghost pixel sets corresponding to the corresponding edge pixels. Otherwise, it is highly likely that the area where the first neighborhood set corresponding to the edge pixels on edge B is located is the non-ghost area, and it is highly likely that the area where the second neighborhood set corresponding to the edge pixels on edge B is located is the ghost area, then all edge pixels on edge B are recorded as the suspected ghost pixel sets. The first neighborhood sets corresponding to the edge pixels are all recorded as the non-ghosting pixel sets corresponding to the corresponding edge pixels, the second neighborhood sets corresponding to all edge pixels on edge B are all recorded as the suspected ghosting pixel sets corresponding to the corresponding edge pixels, and the second means corresponding to each edge pixel on edge B are recorded as the difference feature means corresponding to the corresponding edge pixel; in addition, the grayscale value distribution of the pixels in the ghost area is more chaotic than that in the non-ghosting area, so based on the first chaotic distribution degree and the second chaotic distribution degree corresponding to edge B, the suspected ghosting pixel set and the non-ghosting pixel set can be analyzed, and the difference feature means corresponding to the edge pixels can be determined.

[0034] In this embodiment, the process of obtaining the first mean and the second mean corresponding to each edge pixel point on edge B is as follows: for edge pixel point b on edge B:

[0035] First, obtain the neighborhood difference mean of each pixel point in the first neighborhood set corresponding to the edge pixel point b and the neighborhood difference mean of each pixel point in the second neighborhood set corresponding to the edge pixel point b; then obtain the mean of the neighborhood difference means of all pixels in the first neighborhood set corresponding to the edge pixel point b, and use it as the first mean corresponding to the edge pixel point b; obtain the mean of the neighborhood difference means of all pixels in the second neighborhood set corresponding to the edge pixel point b, and use it as the second mean corresponding to the edge pixel point b.

[0036] And the method for obtaining the neighborhood difference mean is: the neighborhood difference mean of the c-th pixel point in any neighborhood set C0 refers to the mean of the neighborhood difference value set corresponding to the c-th pixel point, and the d-th neighborhood difference value in the neighborhood difference value set corresponding to the c-th pixel point refers to the absolute value of the grayscale difference between the d-th pixel point and the c-th pixel point in the neighborhood subset corresponding to the c-th pixel point, the neighborhood subset corresponding to the c-th pixel point refers to the set formed by all pixels except the c-th pixel point in the neighborhood set C0, and the number of data in the neighborhood difference value set corresponding to the c-th pixel point is the same as the number of pixels in the neighborhood subset corresponding to the c-th pixel point.

[0037] Therefore, this embodiment can obtain the suspected ghost pixel set, non-ghost pixel set and difference feature mean corresponding to each edge pixel on each edge through the above process; and because the blurring phenomenon in the image caused by the movement of the object will flatten the difference between adjacent pixels, the color and brightness in the edge area of ​​the moving object will no longer show a sudden change, but will be presented in a gentle transition manner. Therefore, this embodiment will then, based on the above-described features, obtain the motion blur degree representation value corresponding to each frame image according to the suspected ghost pixel set, non-ghost pixel set and difference feature mean corresponding to each edge pixel on each edge of each frame image. The specific acquisition process is:

[0038] First, obtain the motion ghost edge possibility index value corresponding to each edge on each frame image in the visual data to be analyzed, and the larger the value of the motion ghost edge possibility index value, the greater the possibility that the corresponding edge is a motion ghost edge, that is, the more the corresponding edge has the characteristics of a motion ghost edge; then, according to the motion ghost edge possibility index value corresponding to each edge on each frame image, obtain the motion blur degree characterization value corresponding to each frame image; and for ease of understanding, this embodiment will take the acquisition process of the motion blur degree characterization value corresponding to any frame image F in the visual data to be analyzed as an example to describe, that is, the acquisition process of the motion blur degree characterization value corresponding to image F is: first, among all the edges on image F, obtain all edges whose motion ghost edge possibility index value is not less than a preset possibility threshold, and record them as motion ghost edges, that is, the edges whose motion ghost edge possibility index value is not less than the preset possibility threshold are motion ghost edges, and the motion ghost edges refer to the edges that separate the ghost area from other areas, and the ghost area is a ghosting area of ​​a moving object in an image, and the ghosting area is caused by the movement of the object; then the number of edge pixels on each moving ghost edge on the image F is obtained, and recorded as the length representation value corresponding to the corresponding moving ghost edge, and then the product of the moving ghost edge possibility index value corresponding to each moving ghost edge on the image F and the length representation value corresponding to the corresponding moving ghost edge is calculated, and recorded as the target representation value of the corresponding moving ghost edge; then the cumulative sum of the target representation values ​​of all moving ghost edges on the image F is obtained, and recorded as the motion blur degree representation value corresponding to the image F; and the larger the motion blur degree representation value corresponding to the image F is, the higher the blur degree of the image F caused by the movement of the object is, that is, the more serious the blur phenomenon of the image F is, and vice versa, it indicates that the blur degree of the image F caused by the movement of the object is lower; in addition, when the length of the moving ghost edge on the image F is longer and the corresponding motion ghost edge possibility index value is larger, it indicates that the motion blur degree representation value corresponding to the image F is larger.

[0039] In addition, in a specific application, the implementer needs to set the preset possibility threshold according to the actual situation. For example, in this embodiment, the preset possibility threshold can be set to 0.6.

[0040] In this embodiment, the process of obtaining the motion ghost edge possibility index value corresponding to each edge on each frame image in the visual data to be analyzed is as follows:

[0041] For edge B: first, obtain the grayscale feature difference mean corresponding to each edge pixel on edge B, and take the inverse of the cumulative sum of the grayscale feature difference means corresponding to all edge pixels on edge B as the first eigenvalue corresponding to edge B; then obtain the first ratio corresponding to each edge pixel on edge B, and then obtain the product of the difference feature mean corresponding to each edge pixel on edge B and the first ratio corresponding to the corresponding edge pixel, and record it as the first representation value of the corresponding edge pixel; then obtain the first representation values ​​of all edge pixels on edge B and accumulate them, and normalize the result obtained after the accumulation, and record the normalized result as the second eigenvalue corresponding to edge B; finally, add the first eigenvalue corresponding to edge B and the corresponding second eigenvalue, and record the addition result as the motion ghost edge possibility index value corresponding to edge B;

[0042] In this embodiment, the method for obtaining the grayscale feature difference mean corresponding to the edge pixel point is: for any edge pixel point, obtain the first grayscale mean and the second grayscale mean of the edge pixel point, and use the absolute value of the difference between the first grayscale mean of the edge pixel point and its corresponding second grayscale mean as the grayscale feature difference mean corresponding to the edge pixel point, and the first grayscale mean of the edge pixel point is the grayscale mean of all pixels in the non-ghost pixel point set corresponding to the edge pixel point, and the second grayscale mean of the edge pixel point is the grayscale mean of all pixels in the ghost pixel point set corresponding to the edge pixel point.

[0043] And the method for obtaining the first ratio corresponding to any edge pixel point b on edge B is: in the non-ghosting pixel point set corresponding to the edge pixel point b, obtain the grayscale value with the largest number of occurrences, and record it as the first grayscale value; then obtain the number of times the first grayscale value appears in the suspected ghosting pixel point set corresponding to the edge pixel point b, and record it as the first frequency value; then, the ratio of the first frequency value to the total number of pixels in the suspected ghosting pixel point set corresponding to the edge pixel point b is recorded as the first ratio corresponding to the edge pixel point b; and if the number of first grayscale values ​​is greater than 1, then when obtaining the number of times the first grayscale value appears in the suspected ghosting pixel point set corresponding to the edge pixel point b, one first grayscale value can obtain a frequency statistical result, and this embodiment uses the maximum value as the first frequency value.

[0044] In addition, the specific expression for obtaining the motion ghost edge possibility index value corresponding to edge B is:

[0045]

[0046] in, is the motion ghost edge possibility index value corresponding to edge B, is the number of edge pixels on edge B, is the mean gray feature difference corresponding to the nth edge pixel on edge B, is the first eigenvalue corresponding to edge B, Norm() is the normalization function, is the mean value of the difference feature corresponding to the nth edge pixel on edge B, is the first ratio corresponding to the nth edge pixel on edge B, is the first representation value of the nth edge pixel on edge B, is the second eigenvalue corresponding to edge B.

[0047] And when The smaller the value of is, the smaller the grayscale difference on both sides of edge B is, and the more it conforms to the characteristics of the motion ghost edge. The characteristics of the motion ghost edge here refer to the fact that the mutation characteristics on both sides of the motion ghost edge are not obvious, but are presented in a gentle transition, that is, the grayscale difference on both sides of the motion ghost edge is small, so when The smaller the value of is, the greater the possibility that edge B is a motion ghost edge, that is, The larger the value of The larger the value of The smaller the value of The larger the value of , the greater the possibility that the area where the suspected ghost pixel set corresponding to the edge pixel point on edge B belongs to the ghost area. When the possibility that the area where the suspected ghost pixel set corresponding to the edge pixel point on edge B belongs to the ghost area is greater, it indicates that the possibility that edge B is a motion ghost edge is greater, that is, The larger the value of The smaller the value of The smaller the value of .

[0048] Therefore, this embodiment can obtain the motion blur degree representation value corresponding to each frame image in the visual data to be analyzed through the above process; since in this embodiment, the visual data to be analyzed includes the current frame image, and the current frame image is collected at the current collection time, this embodiment will then combine the motion blur degree representation value corresponding to each frame image in the visual data to be analyzed to obtain the motion blur degree prediction value of the next frame image, and the motion blur degree prediction value of the next frame image can reflect the motion blur degree of the next frame image to be collected, and the collection frame rate of the next frame image will be determined based on the motion blur degree prediction value of the next frame image. Then the process of obtaining the motion blur degree prediction value of the next frame image is as follows:

[0049] A motion blur degree difference sequence is obtained, and the g-th motion blur degree difference in the motion blur degree difference sequence is the result of subtracting the motion blur degree representation value corresponding to the g+1-th frame image in the visual data to be analyzed from the motion blur degree representation value corresponding to the g-th frame image in the visual data to be analyzed; then, in the motion blur degree difference sequence, all motion blur degree differences that are not less than a preset difference threshold are obtained, and all motion blur degree differences that are not less than the preset difference threshold are accumulated, and then the accumulated result is used as the motion blur degree prediction value of the next frame image; and the larger or more motion blur degree differences that are not less than the preset difference threshold, the greater the probability that a blur phenomenon or a ghosting phenomenon exists in the collected image due to the movement of the object when the next frame image is collected, and conversely, the smaller or fewer motion blur degree differences that are not less than the preset difference threshold, the smaller the probability that a blur phenomenon or a ghosting phenomenon exists in the collected image due to the movement of the object when the next frame image is collected, so if the present embodiment is to achieve the purpose of improving the quality of the collected visual data and reducing the generation of redundant visual data, it is necessary to determine the acquisition frame rate of the next frame image according to the motion blur degree prediction value.

[0050] In a specific application, the implementer needs to set the preset difference threshold according to the actual situation. For example, in this embodiment, the preset difference threshold is set to 0.

[0051] Step S003, obtaining a capture frame rate of a next frame of image according to the motion blur degree prediction value, and acquiring the next frame of image at the capture frame rate of the next frame of image.

[0052] This embodiment will obtain the acquisition frame rate of the next frame of image based on step S002, and the specific process is as follows:

[0053] First, determine whether there is no motion ghost edge in the current frame image. If not, set the acquisition frame rate of the next frame image to the preset acquisition frame rate, and this can reduce redundant data. Otherwise, obtain the adjustment degree, and obtain the acquisition frame rate of the next frame image according to the adjustment degree, that is, when the next frame image is subsequently acquired, it will be acquired at the acquisition frame rate of the next frame image acquired at this time, and determining the acquisition frame rate of the next frame image according to the adjustment degree can improve the acquisition quality of the next frame image. The adjustment degree is the result of multiplying the normalized value of the motion blur degree characterization value corresponding to the current frame image by the normalized value of the motion blur degree prediction value of the next frame image, and in this embodiment, the normalization function Norm() is used for normalization.

[0054] In this embodiment, according to the adjustment degree, the process of obtaining the acquisition frame rate of the next frame image is as follows: the acquisition frame rate when acquiring the current frame image is obtained and recorded as the current acquisition frame rate, and then the preset first constant is added to the adjustment degree and the result is multiplied by the current acquisition frame rate, which is recorded as the acquisition frame rate of the next frame image; and when obtaining the acquisition frame rate of the next frame image, in order to control the adjustment rate, a control factor can also be added, such as the product of the adjustment degree and the control factor, which is recorded as the target adjustment degree, and the result of adding the preset first constant to the target adjustment degree and multiplying the result by the current acquisition frame rate is recorded as the acquisition frame rate of the next frame image; and the greater the adjustment degree, the greater the difference between the acquisition frame rate of the next frame image and the current acquisition frame rate.

[0055] In addition, in a specific application, the implementer needs to set the preset first constant and the control factor according to the actual situation. For example, in this embodiment, the preset first constant and the control factor are set to 1 and 0.5 respectively.

[0056] Therefore, this embodiment can obtain the acquisition frame rate of the next frame image through the above process.

[0057] In summary, the present embodiment first obtains the visual data to be analyzed collected by the target acquisition device; then performs edge detection on each frame image in the visual data to be analyzed, obtains all edges in each frame image, obtains the first neighborhood set and the second neighborhood set corresponding to each edge pixel point on the edge according to the pixel points on both sides of the edge, obtains the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean value corresponding to each edge pixel point on the edge according to the gray value of the pixel points in the first neighborhood set and the second neighborhood set, obtains the motion blur degree characterization value corresponding to each frame image according to the suspected ghost pixel point set, the non-ghost pixel point set and the difference feature mean value, obtains the motion blur degree characterization value corresponding to each frame image according to the motion blur degree characterization value corresponding to each frame image, obtains the motion blur degree prediction value of the next frame image according to the motion blur degree prediction value; finally, obtains the acquisition frame rate of the next frame image according to the motion blur degree prediction value, and acquires the next frame image at the acquisition frame rate of the next frame image. The acquisition frame rate of the next frame image obtained in the present embodiment based on the motion blur degree characterization value corresponding to each frame image can improve the quality of the acquired visual data and reduce the generation of redundant visual data.

[0058] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A visual data intelligent acquisition system based on the Internet of Things, comprising a processor and a memory, characterized in that: The processor executes the computer program stored in the memory to implement the following steps: Acquire visual data to be analyzed collected by a target acquisition device, wherein the visual data to be analyzed consists of A frame images, where A is a positive integer greater than 0, and the visual data to be analyzed includes a current frame image collected at a current acquisition time; Performing edge detection on each frame image in the visual data to be analyzed to obtain all edges in each frame image, and obtaining a first neighborhood set and a second neighborhood set corresponding to each edge pixel point on the edge according to pixel points on both sides of the edge; According to the grayscale values ​​of the pixels in the first neighborhood set and the second neighborhood set, obtaining a suspected ghost pixel set, a non-ghost pixel set, and a difference feature mean corresponding to each edge pixel on the edge; According to the suspected ghost pixel set, the non-ghost pixel set and the difference feature mean, a motion blur degree representation value corresponding to each frame image is obtained; according to the motion blur degree representation value corresponding to each frame image, a motion blur degree prediction value of the next frame image is obtained; According to the motion blur degree prediction value, the acquisition frame rate of the next frame image is obtained, and the next frame image is acquired by acquiring the acquisition frame rate of the next frame image; The method for obtaining the motion blur degree representation value corresponding to each frame image includes: obtaining the motion ghost edge possibility index value corresponding to the edge; for any frame image F in the visual data to be analyzed, among all the edges on the image F, all edges whose motion ghost edge possibility index value is not less than a preset possibility threshold are recorded as motion ghost edges, the number of edge pixels on each motion ghost edge on the image F is recorded as the length representation value corresponding to the corresponding motion ghost edge, the product of the motion ghost edge possibility index value corresponding to each motion ghost edge on the image F and the length representation value corresponding to the corresponding motion ghost edge is recorded as the target representation value of the corresponding motion ghost edge, and the cumulative sum of the target representation values ​​of all motion ghost edges on the image F is recorded as the motion blur degree representation value corresponding to the image F; The method for obtaining the motion ghost edge possibility index value corresponding to the edge includes: for any edge B: obtaining the grayscale feature difference mean value corresponding to each edge pixel point on the edge B, recording the inverse of the cumulative sum of the grayscale feature difference means corresponding to all edge pixel points on the edge B as the first eigenvalue corresponding to the edge B; obtaining the first ratio value corresponding to each edge pixel point on the edge B, recording the product of the difference feature mean value corresponding to each edge pixel point on the edge B and the first ratio value corresponding to the corresponding edge pixel point as the first characterization value of the corresponding edge pixel point; recording the normalized value of the result obtained by accumulating the first characterization values ​​of all edge pixel points on the edge B as the second eigenvalue corresponding to the edge B; recording the sum of the first eigenvalue and the second eigenvalue corresponding to the edge B as the motion ghost edge possibility index value corresponding to the edge B; The grayscale feature difference mean corresponding to the edge pixel point is the absolute value of the difference between the first grayscale mean and the second grayscale mean of the corresponding edge pixel point, the first grayscale mean of the edge pixel point is the grayscale mean of the non-ghosting pixel point set corresponding to the corresponding edge pixel point, and the second grayscale mean of the edge pixel point is the grayscale mean of the ghosting pixel point set corresponding to the corresponding edge pixel point; the method for obtaining the first ratio corresponding to any edge pixel point b on the edge B is as follows: in the non-ghosting pixel point set corresponding to the edge pixel point b, obtain the grayscale value with the largest number of occurrences, and record it as the first grayscale value; obtain the number of occurrences of the first grayscale value in the suspected ghosting pixel point set corresponding to the edge pixel point b, and record it as the first frequency value; the ratio of the first frequency value to the total number of pixels in the suspected ghosting pixel point set corresponding to the edge pixel point b is recorded as the first ratio corresponding to the edge pixel point b; The method for acquiring the acquisition frame rate of the next frame image comprises: determining whether there is no motion ghost edge in the current frame image, if not, using the preset acquisition frame rate as the acquisition frame rate of the next frame image, if yes, acquiring the adjustment degree, and acquiring the acquisition frame rate of the next frame image according to the adjustment degree, wherein the adjustment degree refers to the result of multiplying the normalized value of the motion blur degree representation value corresponding to the current frame image by the normalized value of the motion blur degree prediction value; A method for obtaining the acquisition frame rate of the next frame image according to the adjustment degree includes: obtaining the acquisition frame rate when acquiring the current frame image and recording it as the current acquisition frame rate, adding a preset first constant to the adjustment degree and then multiplying the result by the current acquisition frame rate, and recording it as the acquisition frame rate of the next frame image.

2. The visual data intelligent collection system based on the Internet of Things as claimed in claim 1, characterized in that: The method for acquiring the first neighborhood set and the second neighborhood set corresponding to each edge pixel point on the edge includes: For any edge pixel point b on any edge B: obtain the first neighborhood window and the second neighborhood window corresponding to the edge pixel point b; record the set constructed by all pixel points in the first neighborhood window that do not belong to the edge B as the first neighborhood set corresponding to the edge pixel point b; record the set constructed by all pixel points in the second neighborhood window that do not belong to the edge B as the second neighborhood set corresponding to the edge pixel point b.

3. The visual data intelligent collection system based on the Internet of Things as claimed in claim 2, characterized in that: The method for acquiring the first neighborhood window and the second neighborhood window corresponding to the edge pixel point b includes: A tangent line of edge pixel point b located on edge B is obtained and recorded as a first tangent line; a straight line perpendicular to the first tangent line is drawn through edge pixel point b and recorded as a first vertical line; two pixel points located on the first vertical line and closest to edge pixel point b are recorded as a first pixel point and a second pixel point respectively; a nearest neighbor pixel point and a nearest neighbor pixel point of the first pixel point are obtained and recorded as a first feature point and a second feature point of the edge pixel point b respectively, the first feature point and the second feature point are both located on the first vertical line and are not edge pixel point b; two windows are constructed with the first feature point and the second feature point as center points respectively, and are recorded as a first neighborhood window and a second neighborhood window corresponding to edge pixel point b respectively; a first curve corresponding to edge B does not intersect with a second curve corresponding to edge B, the first curve is a curve obtained by connecting the center points of the first neighborhood windows corresponding to all edge pixels on edge B, and the second curve is a curve obtained by connecting the center points of the second neighborhood windows corresponding to all edge pixels on edge B.

4. The visual data intelligent collection system based on the Internet of Things as claimed in claim 1, characterized in that: The method for obtaining the suspected ghost pixel set, the non-ghost pixel set and the difference feature mean corresponding to each edge pixel on the edge includes: For any edge B: obtain the first mean and the second mean corresponding to each edge pixel point on the edge B; record the cumulative sum of the first mean values ​​corresponding to all edge pixel points on the edge B as the first chaotic distribution degree corresponding to the edge B, and record the cumulative sum of the second mean values ​​corresponding to all edge pixel points on the edge B as the second chaotic distribution degree corresponding to the edge B; determine whether the first chaotic distribution degree is greater than the second chaotic distribution degree, and if so, record the first neighborhood sets corresponding to all edge pixel points on the edge B as the suspected ghost pixel set corresponding to the corresponding edge pixel points, and record the first neighborhood set corresponding to all edge pixel points on the edge B as the suspected ghost pixel set corresponding to the corresponding edge pixel points. The first mean corresponding to the edge pixel points on edge B is recorded as the difference feature mean corresponding to the corresponding edge pixel points, and the second neighborhood sets corresponding to all edge pixel points on edge B are recorded as the non-ghosting pixel set corresponding to the corresponding edge pixel points; otherwise, the first neighborhood sets corresponding to all edge pixel points on edge B are recorded as the non-ghosting pixel set corresponding to the corresponding edge pixel points, the second neighborhood sets corresponding to all edge pixel points on edge B are recorded as the suspected ghosting pixel set corresponding to the corresponding edge pixel points, and the second mean corresponding to the edge pixel points on edge B is recorded as the difference feature mean corresponding to the corresponding edge pixel points.

5. The visual data intelligent collection system based on the Internet of Things as claimed in claim 4, characterized in that: The method for obtaining the first mean and the second mean corresponding to each edge pixel point on the edge B comprises: For any edge pixel point b on the edge B: obtain the neighborhood difference mean of each pixel point in the first neighborhood set corresponding to the edge pixel point b and the neighborhood difference mean of each pixel point in the second neighborhood set corresponding to the edge pixel point b; take the mean of the neighborhood difference means of all pixels in the first neighborhood set corresponding to the edge pixel point b as the first mean corresponding to the edge pixel point b, take the mean of the neighborhood difference means of all pixels in the second neighborhood set corresponding to the edge pixel point b as the second mean corresponding to the edge pixel point b, the neighborhood difference mean of the c-th pixel point in the neighborhood set refers to the mean of the neighborhood difference value set corresponding to the c-th pixel point, the d-th neighborhood difference value in the neighborhood difference value set corresponding to the c-th pixel point refers to the absolute value of the grayscale difference between the d-th pixel point in the neighborhood subset corresponding to the c-th pixel point and the c-th pixel point, and the neighborhood subset corresponding to the c-th pixel point refers to the set formed by all pixels except the c-th pixel point in the neighborhood set where the c-th pixel point is located.

6. The visual data intelligent collection system based on the Internet of Things as claimed in claim 1, characterized in that: The method for obtaining the predicted value of the motion blur level of the next frame image includes: A motion blur degree difference sequence is obtained, wherein the g-th motion blur degree difference value in the motion blur degree difference sequence is the result of subtracting the motion blur degree representation value corresponding to the g-th frame image in the visual data to be analyzed from the motion blur degree representation value corresponding to the g+1-th frame image in the visual data to be analyzed; in the motion blur degree difference sequence, the sum of all motion blur degree differences that are not less than a preset difference threshold is added up as the motion blur degree prediction value for the next frame image.

Images