Automatic Warning Method, Device, Equipment and Storage Medium Based on Image Recognition

Abstract

The present invention relates to artificial intelligence and digital medical technologies, and discloses an automatic warning method based on image recognition, including: performing frequency domain conversion and filtering processing on an original monitoring image to obtain an initial monitoring image, binarizing the initial monitoring image according to a calculated global threshold to obtain a standard monitoring image, extracting a standard skeleton image from the standard monitoring image, screening multiple skeleton lines in the standard skeleton image, and performing intersection processing on the multiple skeleton lines to obtain multiple skeleton key points, mapping the multiple skeleton key points to a preset coordinate system to obtain multiple target key regions and identifying the number of regions, and giving a warning according to the number and a preset warning rule. In addition, the present invention also relates to blockchain technology, and the global threshold can be stored in the nodes of the blockchain. The present invention also provides an automatic warning device, an electronic device, and a storage medium based on image recognition. The present invention can solve the problem of relatively low accuracy of automatic warning.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to an automatic warning method, device, electronic device and computer-readable storage medium based on image recognition. Background Art

[0002] With the rapid development of the Internet and the economy, people's work and life are busier. In order to save time, it is usually hoped that the queuing time can be reduced in various queuing occasions, so that more time can be spent on work. Therefore, there is an urgent need for an automatic warning method when the queue is congested.

[0003] The existing automatic warning methods usually install hardware that can count on the door of the venue. When someone enters the door, counting will be carried out, and an alarm will be issued when the count reaches a certain threshold. However, this method does not consider the repeated counting caused by entering and leaving the door, resulting in a low accuracy of automatic warning. Summary of the Invention

[0004] The present invention provides an automatic warning method, device and computer-readable storage medium based on image recognition, and its main purpose is to solve the problem of low accuracy of automatic warning.

[0005] To achieve the above object, an automatic warning method based on image recognition provided by the present invention includes:

[0006] Obtain an original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image;

[0007] Calculate the global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image;

[0008] Extract a standard skeleton image from the standard monitoring image according to preset skeleton extraction conditions;

[0009] Screen multiple skeleton lines in the standard skeleton image, and perform intersection processing on the multiple skeleton lines to obtain multiple skeleton key points;

[0010] Map the multiple skeleton key points to a preset coordinate system to obtain multiple target key areas and identify the number of the multiple target key areas;

[0011] Perform warning according to the number of the multiple target key areas and a preset warning rule.

[0012] Optionally, the calculating the global threshold of the initial monitoring image includes:

[0013] Assign the pixel points in the initial monitoring image to each block in a preset histogram, and count the number of pixel points contained in each block in the histogram;

[0014] Divide the number of pixel points contained in each block by the total number of pixel points in the initial monitoring image respectively to obtain the block value corresponding to each block;

[0015] Obtain a preset classification value, a first threshold, and a second threshold, construct a first interval with the classification value and the first threshold, and construct a second interval with the classification value and the second threshold;

[0016] Let the pixel points with the block value in the first interval be foreground pixels, calculate the ratio of the foreground pixels to the total number of pixel points in the initial monitoring image as the foreground ratio, and obtain the foreground gray level corresponding to the foreground pixels;

[0017] Let the pixel points with the block value in the second interval be background pixels, calculate the ratio of the background pixels to the total number of pixel points in the initial monitoring image as the background ratio, and obtain the background gray level corresponding to the background pixels;

[0018] Calculate the variance value of the foreground pixels and the background pixels using a preset variance formula;

[0019] Reset multiple classification values, construct corresponding intervals respectively according to the set multiple classification values, the first threshold, and the second threshold, and execute the step of calculating the variance value. Sort the variance values corresponding to the multiple classification values from largest to smallest to obtain a variance value ranking list;

[0020] Take the classification value corresponding to the variance value ranked first in the variance value ranking list as the global threshold.

[0021] Optionally, the binary processing of the initial monitoring image according to the global threshold to obtain a standard monitoring image includes:

[0022] Set the pixel points in the initial monitoring image that are greater than the global threshold to a preset first gray level;

[0023] Set the pixel points in the initial monitoring image that are less than or equal to the global threshold to a preset second gray level to obtain a standard monitoring image.

[0024] Optionally, the extraction of the standard skeleton image from the standard region image according to the preset skeleton extraction conditions includes:

[0025] Delete the pixel points in the standard region image that meet the first skeleton extraction condition, where the first skeleton extraction condition is that the product of multiple target pixel points that satisfy a preset first product formula is a first preset value; or

[0026] Delete the pixel points in the standard region image that meet the second skeleton extraction condition to obtain a standard skeleton image, where the second skeleton extraction condition is that the product of multiple target pixel points that satisfy a preset second product formula is a second preset value.

[0027] Optionally, screening multiple skeleton straight lines in the standard skeleton image includes:

[0028] Detect multiple straight lines in the standard skeleton image using a preset straight line detection algorithm;

[0029] Delete the straight lines with a straight line length less than a preset straight line threshold among the multiple straight lines to obtain multiple skeleton straight lines.

[0030] Optionally, performing frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image includes:

[0031] Perform spatial conversion processing on the original monitoring image to obtain a frequency domain image;

[0032] Perform filtering processing on the frequency domain image using a preset filtering function to obtain a filtered image;

[0033] Perform spatial restoration processing on the filtered image to obtain an initial monitoring image.

[0034] Optionally, performing spatial conversion processing on the original monitoring image to obtain a frequency domain image includes:

[0035] Perform spatial conversion processing on the original monitoring image using a preset fast Fourier formula to obtain a frequency domain image:

[0036]

[0037] Among them, f(x,y) represents the pixel value of the original monitoring image, F(u,v) represents the pixel value of the frequency domain image, M and N represent the width and height of the original monitoring image, j is a fixed parameter in the fast Fourier transform function, x and y respectively represent the x-th row and y-th column in the original monitoring image, and u and v respectively represent the u-th row and v-th column in the frequency domain image.

[0038] To solve the above problems, the present invention also provides an automatic warning device based on image recognition, and the device includes:

[0039] A frequency domain conversion module, which is used to obtain an original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image;

[0040] A binarization module, which is used to calculate a global threshold of the initial monitoring image and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image;

[0041] A skeleton extraction module, which is used to extract a standard skeleton image from the standard monitoring image according to preset skeleton extraction conditions;

[0042] A straight line intersection module, which is used to screen multiple skeleton straight lines in the standard skeleton image and perform intersection processing on the multiple skeleton straight lines to obtain multiple skeleton key points;

[0043] A key area acquisition module, which is used to map the multiple skeleton key points to a preset coordinate system to obtain multiple target key areas and identify the number of the multiple target key areas;

[0044] An early warning module, which is used to give an early warning according to the number of the multiple target key areas and a preset early warning rule.

[0045] To solve the above problems, the present invention also provides an electronic device, which includes:

[0046] At least one processor; and,

[0047] A memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned automatic early warning method based on image recognition.

[0049] To solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is executed by a processor in an electronic device to implement the above-mentioned automatic early warning method based on image recognition.

[0050] In an embodiment of the present invention, the original monitoring image is subjected to frequency domain conversion and filtering processing to obtain an initial monitoring image. The filtering processing can filter out low-frequency components and high-frequency components. The global threshold of the initial monitoring image is calculated, and the initial monitoring image is binarized according to the global threshold, so as to highlight the contour of the target object in the initial monitoring image. The standard skeleton image is extracted from the standard monitoring image according to the preset skeleton extraction conditions, the accuracy of skeleton screening is improved, and screening, intersection and mapping processing are performed to obtain the target key area. An early warning is given according to the number of the multiple target key areas and the preset early warning rules, and the accuracy of the early warning is improved. Therefore, the automatic early warning method, device, electronic device and computer-readable storage medium based on image recognition proposed by the present invention can solve the problem of low accuracy of automatic early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic flowchart of an automatic early warning method based on image recognition provided by an embodiment of the present invention;

[0052] Figure 2 It is a functional module diagram of an automatic early warning device based on image recognition provided by an embodiment of the present invention;

[0053] Figure 3 It is a schematic structural diagram of an electronic device for implementing the automatic early warning method based on image recognition provided by an embodiment of the present invention.

[0054] The implementation, functional features and advantages of the object of the present invention will be further described in conjunction with the embodiments with reference to the drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] An embodiment of the present application provides an automatic warning method based on image recognition. The execution subject of the automatic warning method based on image recognition includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the automatic warning method based on image recognition can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0057] Referring to Figure 1 As shown, it is a schematic flowchart of an automatic warning method based on image recognition provided by an embodiment of the present invention. In this embodiment, the automatic warning method based on image recognition includes:

[0058] S1. Obtain an original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image.

[0059] In the embodiment of the present invention, the original monitoring image refers to a monitoring image of a queuing line taken in a cafeteria scene, where the queuing line in the original monitoring image includes multiple queuers.

[0060] Specifically, the performing frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image includes:

[0061] Perform spatial transformation processing on the original monitoring image to obtain a frequency domain image;

[0062] Use a preset filtering function to perform filtering processing on the frequency domain image to obtain a filtered image;

[0063] Perform spatial restoration processing on the filtered image to obtain an initial monitoring image.

[0064] In detail, the spatial transformation processing refers to converting the original monitoring image from the spatial domain to the frequency domain, and the spatial transformation can be realized by using the fast Fourier formula. The spatial restoration processing refers to converting the filtered image from the frequency domain to the spatial domain, and the spatial restoration can be realized by using the Fourier inverse transform formula. The filtering processing is to use a Gaussian bandpass filtering function to filter out the low-frequency components and high-frequency components in the frequency domain image. Among them, light interference such as ambient light belongs to the low-frequency components, and noise belongs to the high-frequency components.

[0065] Further, the spatial transformation processing of the original monitoring image to obtain a frequency-domain image includes:

[0066] Performing spatial transformation processing on the original monitoring image using a preset fast Fourier formula to obtain a frequency-domain image:

[0067]

[0068] where f(x, y) represents the pixel value of the original monitoring image, F(u, v) represents the pixel value of the frequency-domain image, M and N represent the width and height of the original monitoring image, j is a fixed parameter in the fast Fourier transform function, x and y respectively represent the x-th row and y-th column in the original monitoring image, and u and v respectively represent the u-th row and v-th column in the frequency-domain image.

[0069] Specifically, in the embodiments of the present invention, the following filtering function is used to perform filtering processing on the frequency-domain image to obtain a filtered image:

[0070]

[0071] where H(u, v) is the pixel value of the filtered image, F(u, v) is the pixel value of the frequency-domain image, D 0 , W, and n are fixed parameters.

[0072] Preferably, n takes a value of 3, D0 takes a value of 120, and W takes a value of 10.

[0073] Further, in the embodiments of the present invention, the following preset Fourier inverse transform formula is used to perform spatial restoration processing on the filtered image to obtain an initial monitoring image:

[0074]

[0075] where L(a, b) is the pixel value of the initial monitoring image, X and Y represent the width and height of the filtered image, j is a fixed parameter in the Fourier inverse transform function, H(u, v) is the pixel value of the filtered image, and a and b are fixed parameters.

[0076] S2. Calculate the global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image.

[0077] In the embodiments of the present invention, performing binarization processing on the initial monitoring image can highlight the contour of the target object.

[0078] Specifically, the calculation of the global threshold of the initial monitoring image includes:

[0079] Assign the pixel points in the initial monitoring image to each block in a preset histogram, and count the number of pixel points contained in each block in the histogram;

[0080] Divide the number of pixel points contained in each block by the total number of pixel points in the initial monitoring image respectively to obtain the block value corresponding to each block;

[0081] Obtain a preset classification value, a first threshold, and a second threshold, construct a first interval with the classification value and the first threshold, and construct a second interval with the classification value and the second threshold;

[0082] Let the pixel points with the block value in the first interval be foreground pixels, calculate the ratio of the foreground pixels to the total number of pixel points in the initial monitoring image as the foreground ratio, and obtain the foreground gray level corresponding to the foreground pixels;

[0083] Let the pixel points with the block value in the second interval be background pixels, calculate the ratio of the background pixels to the total number of pixel points in the initial monitoring image as the background ratio, and obtain the background gray level corresponding to the background pixels;

[0084] Calculate the variance value of the foreground pixels and the background pixels using a preset variance formula;

[0085] Reset multiple classification values, construct corresponding intervals respectively according to the set multiple classification values, the first threshold, and the second threshold, and execute the step of calculating the variance value. Sort the variance values corresponding to the multiple classification values from largest to smallest to obtain a variance value ranking list;

[0086] Take the classification value corresponding to the variance value ranked first in the variance value ranking list as the global threshold.

[0087] Specifically, the embodiment of the present invention calculates the variance value of the foreground pixels and the background pixels using the following variance formula:

[0088] g = w0 * w1 * (μ0 - μ1)(μ0 - μ1)

[0089] Where g is the variance value, w0 is the foreground ratio, w1 is the background ratio, μ0 is the foreground gray level, and μ1 is the background gray level.

[0090] In detail, the preset histogram contains 256 intervals, the first threshold is greater than the classification value, and the classification value is greater than the second threshold.

[0091] Further, the binarization processing of the initial monitoring image according to the global threshold to obtain a standard monitoring image includes:

[0092] Set the pixel points in the initial monitoring image that are greater than the global threshold to a preset first gray value;

[0093] Set the pixel points in the initial monitoring image that are less than or equal to the global threshold to a preset second gray value to obtain a standard monitoring image.

[0094] Specifically, perform binarization processing on the initial monitoring image, set the pixel points in the initial monitoring image that are greater than the global threshold to a preset first gray value, and set the pixel points in the initial monitoring image that are less than or equal to the global threshold to a preset second gray value. For example, convert the pixel points greater than the global threshold to 255, and convert the pixel points less than or equal to the global threshold to 0, so that the area of the pixel points greater than the global threshold is white and other areas are black.

[0095] S3. Extract a standard skeleton image from the standard monitoring image according to preset skeleton extraction conditions.

[0096] In the embodiment of the present invention, the preset skeleton extraction conditions include a first skeleton extraction condition and a second skeleton extraction condition, and extracting a standard skeleton image from the standard region image according to the preset skeleton extraction conditions includes:

[0097] Delete the pixel points in the standard region image that meet the first skeleton extraction condition, where the first skeleton extraction condition is that the product of multiple target pixel points that meet a preset first product formula is a first preset value; or

[0098] Delete the pixel points in the standard region image that meet the second skeleton extraction condition to obtain a standard skeleton image, where the second skeleton extraction condition is that the product of multiple target pixel points that meet a preset second product formula is a second preset value.

[0099] Further, before extracting a standard skeleton image from the standard region image according to the preset skeleton extraction conditions, the method further includes:

[0100] Taking the pixel point directly above the central pixel point in the standard region image as the starting search point, perform a neighborhood search around the first pixel point in a "return" shape to obtain a set of neighborhood pixel points of the first pixel point.

[0101] Specifically, in the embodiment of the present invention, the preset multiple target pixel points are respectively the second pixel, the fourth pixel, the sixth pixel, and the eighth pixel, the first preset value is zero, and the first product formula is the multiplication of the second pixel, the fourth pixel, and the sixth pixel and the multiplication of the fourth pixel, the sixth pixel, and the eighth pixel. Express the first skeleton extraction condition with a formula as:

[0102]

[0103] Among them, P 1 is the first pixel point, S(P 1 ) is the number of non-zero neighboring points of the first pixel point, and S(P 1 ) is the number of times the values of these points change from 0 to 1 in the order of P 2 , P 3 , …, P 9 .

[0104] Furthermore, in the embodiment of the present invention, the second preset value is zero, and the second product formula is the multiplication of the second pixel, the fourth pixel, and the eighth pixel and the multiplication of the second pixel, the sixth pixel, and the eighth pixel. The second skeleton extraction condition is expressed by the formula as follows:

[0105]

[0106] Among them, P 2 is the second pixel point, P 4 is the fourth pixel point, P 6 is the sixth pixel point, and P 8 is the eighth pixel point.

[0107] Specifically, the set of neighboring pixel points of the first pixel point P 1 includes a series of pixel points such as the second pixel point P 2 , the third pixel point P 3 , the fourth pixel point P 4 , etc. There are a total of eight pixel points in this solution.

[0108] S4. Screen multiple skeleton lines in the standard skeleton image, and perform intersection processing on the multiple skeleton lines to obtain multiple skeleton key points.

[0109] In the embodiment of the present invention, the screening of multiple skeleton lines in the standard skeleton image includes:

[0110] Detect multiple lines in the standard skeleton image by using a preset line detection algorithm;

[0111] Delete the lines with a line length less than a preset line threshold among the multiple lines to obtain multiple skeleton lines.

[0112] Specifically, the preset line detection algorithm can be the LSD algorithm or the Hough transform line detection algorithm.

[0113] Further, in the embodiment of the present invention, intersection processing is performed on the multiple skeleton lines to obtain a plurality of skeleton key points. The multiple skeleton lines screened out are subjected to intersection processing according to their original positions, and the intersection points between the lines are the skeleton key points.

[0114] S5. Map the plurality of skeleton key points into a preset coordinate system to obtain a plurality of target key regions and identify the number of the plurality of target key regions.

[0115] In the embodiment of the present invention, the preset coordinate system is a rectangular coordinate system. The plurality of skeleton key points are mapped onto the rectangular coordinate system according to their coordinates to obtain target key regions composed of the plurality of skeleton key points. Among them, the target key region is the target key person's head. Identifying the number of the plurality of target key regions is to identify the number of target key person's heads in the original surveillance image.

[0116] S6. Give an alarm according to the number of the plurality of target key regions and a preset alarm rule.

[0117] In the embodiment of the present invention, giving an alarm according to the number of the plurality of target key regions and a preset alarm rule includes:

[0118] Judge the size between the number of the plurality of target key regions and a preset region threshold;

[0119] When the number of the plurality of target key regions is greater than the region threshold, obtain an interval surveillance image with a preset time interval from the surveillance time of the original surveillance image;

[0120] Identify the key regions in the interval surveillance image and count the number of the key regions;

[0121] Perform a difference calculation process on the number of the key regions and the number of the plurality of target key regions to obtain a region difference;

[0122] When the region difference is less than the preset region threshold, send an alarm warning.

[0123] Specifically, the target key region refers to the recognized person's head in the original surveillance image, and the interval surveillance image refers to an image with a certain time interval from the surveillance time of the original surveillance image. Simply judging whether an alarm is needed based on the number of the target key regions is not comprehensive enough. By performing a difference calculation process on the number of the key regions and the number of the plurality of target key regions to obtain a region difference, the region difference can illustrate the congestion situation. The embodiment of the present invention makes a judgment based on the region difference, improving the accuracy of the alarm.

[0124] In the embodiments of the present invention, through frequency domain conversion and filtering processing of the original monitoring image, an initial monitoring image is obtained. The filtering processing can filter out low-frequency components and high-frequency components. Calculate the global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold, which can highlight the contour of the target object in the initial monitoring image. Extract the standard skeleton image from the standard monitoring image according to the preset skeleton extraction conditions, improve the accuracy of skeleton screening, and perform screening, intersection and mapping processing to obtain the target key area. Issue a warning according to the number of the multiple target key areas and the preset warning rules, which improves the accuracy of the warning. Therefore, the automatic warning method based on image recognition proposed by the present invention can solve the problem of low accuracy of automatic warning.

[0125] As Figure 2 shown, it is a functional module diagram of an automatic warning device based on image recognition provided by an embodiment of the present invention.

[0126] The automatic warning device 100 based on image recognition of the present invention can be installed in an electronic device. According to the functions achieved, the automatic warning device 100 based on image recognition can include a frequency domain conversion module 101, a binarization module 102, a skeleton extraction module 103, a straight line intersection module 104, a key area acquisition module 105 and a warning module 106. The modules of the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0127] In this embodiment, the functions of each module / unit are as follows:

[0128] The frequency domain conversion module 101 is used to obtain the original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image, and obtain an initial monitoring image;

[0129] The binarization module 102 is used to calculate the global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image;

[0130] The skeleton extraction module 103 is used to extract a standard skeleton image from the standard monitoring image according to the preset skeleton extraction conditions;

[0131] The straight line intersection module 104 is used to screen multiple skeleton lines in the standard skeleton image, and perform intersection processing on the multiple skeleton lines to obtain multiple skeleton key points;

[0132] The key area acquisition module 105 is configured to map the multiple skeleton key points into a preset coordinate system, obtain multiple target key areas, and identify the number of the multiple target key areas;

[0133] The warning module 106 is configured to give a warning according to the number of the multiple target key areas and a preset warning rule.

[0134] Specifically, the specific implementation manners of the modules of the automatic warning device 100 based on image recognition are as follows:

[0135] Step 1: Obtain an original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image.

[0136] In the embodiment of the present invention, the original monitoring image refers to a monitoring image of a queuing line taken in a cafeteria scene, where the queuing line in the original monitoring image includes multiple queuers.

[0137] Specifically, the performing frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image includes:

[0138] Perform spatial conversion processing on the original monitoring image to obtain a frequency domain image;

[0139] Perform filtering processing on the frequency domain image by using a preset filtering function to obtain a filtered image;

[0140] Perform spatial restoration processing on the filtered image to obtain an initial monitoring image.

[0141] Specifically, the spatial conversion processing refers to converting the original monitoring image from the spatial domain to the frequency domain, and the spatial conversion can be implemented by using the fast Fourier formula. The spatial restoration processing refers to converting the filtered image from the frequency domain to the spatial domain, and the spatial restoration can be implemented by using the Fourier inverse transform formula. The filtering processing is to filter out the low-frequency components and high-frequency components in the frequency domain image by using a Gaussian band-pass filtering function. Among them, light interference such as ambient light belongs to the low-frequency components, and noise belongs to the high-frequency components.

[0142] Further, the performing spatial conversion processing on the original monitoring image to obtain a frequency domain image includes:

[0143] Perform spatial conversion processing on the original monitoring image by using a preset fast Fourier formula to obtain a frequency domain image:

[0144]

[0145] Among them, f(x, y) represents the pixel value of the original monitored image, F(u, v) represents the pixel value of the frequency-domain image, M and N represent the width and height of the original monitored image, j is a fixed parameter in the fast Fourier transform function, x and y respectively represent the x-th row and y-th column in the original monitored image, and u and v respectively represent the u-th row and v-th column in the frequency-domain image.

[0146] Specifically, the embodiment of the present invention filters the frequency-domain image by using the following filtering function to obtain a filtered image:

[0147]

[0148] Among them, H(u, v) is the pixel value of the filtered image, F(u, v) is the pixel value of the frequency-domain image, D 0 , W, and n are fixed parameters.

[0149] Preferably, n takes the value of 3, D0 takes the value of 120, and W takes the value of 10.

[0150] Further, the embodiment of the present invention performs spatial restoration processing on the filtered image by using the following preset inverse Fourier transform formula to obtain an initial monitored image:

[0151]

[0152] Among them, L(a, b) is the pixel value of the initial monitored image, X and Y represent the width and height of the filtered image, j is a fixed parameter in the inverse Fourier transform function, H(u, v) is the pixel value of the filtered image, and a and b are fixed parameters.

[0153] Step 2: Calculate the global threshold of the initial monitored image, and perform binarization processing on the initial monitored image according to the global threshold to obtain a standard monitored image.

[0154] In the embodiment of the present invention, performing binarization processing on the initial monitored image can highlight the contour of the target object.

[0155] Specifically, the calculation of the global threshold of the initial monitored image includes:

[0156] Assign the pixel points in the initial monitored image to each block in a preset histogram, and count the number of pixel points included in each block in the histogram;

[0157] Respectively divide the number of pixel points included in each block by the total number of pixel points in the initial monitored image to obtain the block value corresponding to each block;

[0158] Obtain a preset classification value, a first threshold, and a second threshold, construct a first interval with the classification value and the first threshold, and construct a second interval with the classification value and the second threshold;

[0159] Let the pixel points of the block value in the first interval be foreground pixels, calculate the ratio of the number of foreground pixels to the total number of pixel points in the initial monitoring image as the foreground ratio, and obtain the foreground gray level corresponding to the foreground pixels;

[0160] Let the pixel points of the block value in the second interval be background pixels, calculate the ratio of the number of background pixels to the total number of pixel points in the initial monitoring image as the background ratio, and obtain the background gray level corresponding to the background pixels;

[0161] Calculate the variance value of the foreground pixels and the background pixels using a preset variance formula;

[0162] Reset multiple classification values, respectively construct corresponding intervals according to the set multiple classification values and the first threshold and the second threshold, and perform the step of calculating the variance value. Sort the variance values corresponding to the multiple classification values from largest to smallest to obtain a variance value ranking list;

[0163] Take the classification value corresponding to the variance value ranked first in the variance value ranking list as the global threshold.

[0164] Specifically, the embodiment of the present invention calculates the variance value of the foreground pixels and the background pixels using the following variance formula:

[0165] g = w0 * w1 * (μ0 - μ1)(μ0 - μ1)

[0166] Where g is the variance value, w0 is the foreground ratio, w1 is the background ratio, μ0 is the foreground gray level, and μ1 is the background gray level.

[0167] In detail, the preset histogram contains 256 intervals, the first threshold is greater than the classification value, and the classification value is greater than the second threshold.

[0168] Further, the binarization processing of the initial monitoring image according to the global threshold to obtain a standard monitoring image includes:

[0169] Set the pixel points in the initial monitoring image that are greater than the global threshold to a preset first gray level value;

[0170] Set the pixel points in the initial monitoring image that are less than or equal to the global threshold to a preset second gray level value to obtain a standard monitoring image.

[0171] Specifically, perform binarization on the initial monitoring image, such that the pixel points in the initial monitoring image that are greater than the global threshold are set to a preset first gray value, and the pixel points in the initial monitoring image that are less than or equal to the global threshold are set to a preset second gray value. For example, convert the pixel points greater than the global threshold to 255, and convert the pixel points less than or equal to the global threshold to 0, so that the area of the pixel points greater than the global threshold is white and other areas are black.

[0172] Step 3: Extract a standard skeleton image from the standard monitoring image according to preset skeleton extraction conditions.

[0173] In the embodiment of the present invention, the preset skeleton extraction conditions include a first skeleton extraction condition and a second skeleton extraction condition, and the extracting of the standard skeleton image from the standard region image according to the preset skeleton extraction conditions includes:

[0174] Delete the pixel points in the standard region image that satisfy the first skeleton extraction condition, where the first skeleton extraction condition is that the product of multiple target pixel points that satisfy a preset first product formula is a first preset value; or

[0175] Delete the pixel points in the standard region image that satisfy the second skeleton extraction condition to obtain a standard skeleton image, where the second skeleton extraction condition is that the product of multiple target pixel points that satisfy a preset second product formula is a second preset value. Further, before extracting the standard skeleton image from the standard region image according to the preset skeleton extraction conditions, the method further includes:

[0176] Using the pixel point directly above the central pixel point in the standard region image as the starting search point, perform a neighborhood search around the first pixel point in a "return" shape to obtain a set of neighborhood pixel points of the first pixel point.

[0177] Specifically, in the embodiment of the present invention, the preset multiple target pixel points are respectively the second pixel, the fourth pixel, the sixth pixel, and the eighth pixel, the first preset value is zero, and the first product formula is the multiplication of the second pixel, the fourth pixel, and the sixth pixel and the multiplication of the fourth pixel, the sixth pixel, and the eighth pixel. Express the first skeleton extraction condition with a formula as:

[0178]

[0179] where P 1 is the first pixel point, S(P 1 ) is the number of non-zero neighboring points of the first pixel point, S(P 1 ) is with P 2 , P 3 , …, P 9It is the number of times the values of these points change from 0 to 1 in chronological order.

[0180] Furthermore, in the embodiment of the present invention, the second preset value is zero, and the second product formula is the multiplication of the second pixel, the fourth pixel, and the eighth pixel and the multiplication of the second pixel, the sixth pixel, and the eighth pixel. The second skeleton extraction condition is expressed by the formula:

[0181]

[0182] where P 2 is the second pixel point, P 4 is the fourth pixel point, P 6 is the sixth pixel point, P 8 is the eighth pixel point.

[0183] Specifically, the neighborhood pixel point set of the first pixel point P 1 includes a series of pixel points such as the second pixel point P 2 , the third pixel point P 3 , the fourth pixel point P 4 , etc. There are a total of eight pixel points in this solution.

[0184] Step Four: Screen multiple skeleton lines in the standard skeleton image and perform intersection processing on the multiple skeleton lines to obtain multiple skeleton key points.

[0185] In the embodiment of the present invention, the screening of multiple skeleton lines in the standard skeleton image includes:

[0186] Using a preset line detection algorithm to detect multiple lines in the standard skeleton image;

[0187] Delete the lines with a line length less than the preset line threshold among the multiple lines to obtain multiple skeleton lines.

[0188] Specifically, the preset line detection algorithm can be the LSD algorithm or the Hough transform line detection algorithm.

[0189] Furthermore, in the embodiment of the present invention, intersection processing is performed on the multiple skeleton lines to obtain multiple skeleton key points. The multiple screened skeleton lines are intersected according to their original positions, and the intersection points between the lines are the skeleton key points.

[0190] Step Five: Map the multiple skeleton key points to a preset coordinate system to obtain multiple target key regions and identify the number of the multiple target key regions.

[0191] In an embodiment of the present invention, the preset coordinate system is a rectangular coordinate system. The multiple skeleton key points are mapped to the rectangular coordinate system according to their coordinates, and a target key area composed of the multiple skeleton key points is obtained. Among them, the target key area is the target key head, and identifying the number of the multiple target key areas is to identify the number of target key heads in the original surveillance image.

[0192] Step Six: Give an alarm according to the number of the multiple target key areas and a preset alarm rule.

[0193] In an embodiment of the present invention, the giving an alarm according to the number of the multiple target key areas and a preset alarm rule includes:

[0194] Judge the size between the number of the multiple target key areas and a preset area threshold;

[0195] When the number of the multiple target key areas is greater than the area threshold, obtain an interval surveillance image with a preset time interval from the surveillance time of the original surveillance image;

[0196] Identify the key area in the interval surveillance image and count the number of the key areas;

[0197] Perform a difference calculation process on the number of the key areas and the number of the multiple target key areas to obtain an area difference;

[0198] When the area difference is less than the preset area threshold, send an alarm warning.

[0199] Specifically, the target key area refers to the recognized human head in the original surveillance image, and the interval surveillance image refers to an image with a certain time interval from the surveillance time of the original surveillance image. Simply judging whether an alarm is needed based on the number of the target key areas is not comprehensive enough. By performing a difference calculation process on the number of the key areas and the number of the multiple target key areas to obtain an area difference, the area difference can illustrate the congestion situation. In an embodiment of the present invention, judgment is made according to the area difference, which improves the accuracy of the alarm.

[0200] In the embodiments of the present invention, the original monitoring image is subjected to frequency domain conversion and filtering processing to obtain an initial monitoring image. The filtering processing can filter out low-frequency components and high-frequency components. The global threshold of the initial monitoring image is calculated, and the initial monitoring image is binarized according to the global threshold, so as to highlight the contour of the target object in the initial monitoring image. The standard skeleton image is extracted from the standard monitoring image according to the preset skeleton extraction conditions, which improves the accuracy of skeleton screening, and screening, intersection and mapping processing are performed to obtain the target key area. Early warning is carried out according to the number of the multiple target key areas and the preset early warning rules, which improves the accuracy of early warning. Therefore, the automatic early warning device based on image recognition proposed by the present invention can solve the problem of low accuracy of automatic early warning.

[0201] As Figure 3 shown, it is a schematic structural diagram of an electronic device for implementing an automatic early warning method based on image recognition provided by an embodiment of the present invention.

[0202] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an automatic early warning program based on image recognition.

[0203] Among them, the processor 10 may be composed of integrated circuits in some embodiments. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connects all components of the entire electronic device through various interfaces and lines, and by running or executing programs or modules stored in the memory 11 (such as executing an automatic early warning program based on image recognition, etc.), and calling data stored in the memory 11, to execute various functions of the electronic device and process data.

[0204] The memory 11 at least includes one type of readable storage medium, which includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 can be an internal storage unit of the electronic device in some embodiments, such as the mobile hard disk of the electronic device. The memory 11 can also be an external storage device of the electronic device in some other embodiments, such as the plug-in mobile hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the electronic device. Further, the memory 11 can also include both the internal storage unit and the external storage device of the electronic device. The memory 11 can be used not only to store the application software installed on the electronic device and various types of data, such as the code of the automatic warning program based on image recognition, etc., but also to temporarily store the data that has been output or will be output.

[0205] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable the connection and communication between the memory 11 and at least one processor 10, etc.

[0206] The communication interface 13 is used for the communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is generally used to establish a communication connection between this electronic device and other electronic devices. The user interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface and a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. Among them, the display can also be appropriately referred to as a display screen or a display unit, and is used to display the information processed in the electronic device and to display the visual user interface.

[0207] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that Figure 3The structures shown do not constitute a limitation on the electronic device 1, and may include fewer or more components than those shown, or combine certain components, or have different component arrangements.

[0208] For example, although not shown, the electronic device may further include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so as to implement functions such as charging management, discharging management, and power consumption management through the power management device. The power source may also include any components such as one or more DC or AC power sources, a recharge device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may also include various sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be elaborated here.

[0209] It should be understood that the embodiments are for illustrative purposes only and are not limited by this structure in the scope of the patent application.

[0210] The automatic warning program based on image recognition stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can implement:

[0211] Obtain the original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image;

[0212] Calculate the global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image;

[0213] Extract a standard skeleton image from the standard monitoring image according to preset skeleton extraction conditions;

[0214] Screen multiple skeleton lines in the standard skeleton image, and perform intersection processing on the multiple skeleton lines to obtain multiple skeleton key points;

[0215] Map the multiple skeleton key points to a preset coordinate system to obtain multiple target key regions and identify the number of the multiple target key regions;

[0216] Perform warning according to the number of the multiple target key regions and a preset warning rule.

[0217] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiments of the attached drawings, which will not be elaborated here.

[0218] Furthermore, if the modules / units integrated in the electronic device 1 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory).

[0219] The present invention also provides a computer-readable storage medium storing a computer program, which when executed by a processor of an electronic device, can implement:

[0220] Obtain an original monitoring image, perform frequency-domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image;

[0221] Calculate a global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image;

[0222] Extract a standard skeleton image from the standard monitoring image according to preset skeleton extraction conditions;

[0223] Screen multiple skeleton lines in the standard skeleton image, and perform intersection processing on the multiple skeleton lines to obtain multiple skeleton key points;

[0224] Map the multiple skeleton key points to a preset coordinate system to obtain multiple target key regions and identify the number of the multiple target key regions;

[0225] Perform early warning according to the number of the multiple target key regions and a preset early warning rule.

[0226] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation.

[0227] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, in each embodiment of the present invention, each functional module can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional modules.

[0229] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms.

[0230] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any associated drawing reference signs in the claims should not be regarded as limiting the claimed rights.

[0231] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, a string of data blocks generated by using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity (anti-counterfeiting) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, an application service layer, etc.

[0232] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use the knowledge to obtain the best results of theory, method, technology, and application system.

[0233] In addition, obviously, the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. Words such as first, second, etc. are used to represent names and do not represent any specific order.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An automatic warning method based on image recognition, characterized in that, the method includes: Obtain the original monitoring image, perform frequency domain conversion and filtering processing on the original monitoring image to obtain an initial monitoring image; Calculate the global threshold of the initial monitoring image, and perform binarization processing on the initial monitoring image according to the global threshold to obtain a standard monitoring image; Delete the pixel points that meet the first preset value or the second preset value in the standard monitoring image according to the preset skeleton extraction conditions to obtain a standard skeleton image. The pixel points of the first preset value or the second preset value are the products of multiple target pixel points that meet the preset first product formula or the second product formula. The multiple target pixel points are respectively the second pixel, the fourth pixel, the sixth pixel and the eighth pixel. The first preset value or the second preset value is zero. The first product formula is the multiplication of the second pixel, the fourth pixel and the sixth pixel and the multiplication of the fourth pixel, the sixth pixel and the eighth pixel. The second product formula is the multiplication of the second pixel, the fourth pixel and the eighth pixel and the multiplication of the second pixel, the sixth pixel and the eighth pixel, a total of eight pixel points; Screen multiple skeleton lines in the standard skeleton image, perform intersection processing on the multiple skeleton lines, and use the intersection points between the lines as skeleton key points to obtain multiple skeleton key points; Map the multiple skeleton key points to a preset coordinate system to obtain multiple target key areas and identify the number of the multiple target key areas; Give a warning according to the number of the multiple target key areas and a preset warning rule.

2. The automatic warning method based on image recognition according to claim 1, characterized in that, the calculation of the global threshold of the initial monitoring image includes: Assign the pixel points in the initial monitoring image to each block in a preset histogram, and count the number of pixel points contained in each block in the histogram; Divide the number of pixel points contained in each block by the total number of pixel points in the initial monitoring image respectively to obtain the block value corresponding to each block; Obtain a preset classification value, a first threshold and a second threshold, construct a first interval with the classification value and the first threshold, and construct a second interval with the classification value and the second threshold; Let the pixel points with the block value in the first interval be foreground pixels, calculate the ratio of the foreground pixels to the total number of pixel points in the initial monitoring image as the foreground ratio, and obtain the foreground gray level corresponding to the foreground pixels; Let the pixel points with the block value in the second interval be background pixels, calculate the ratio of the background pixels to the total number of pixel points in the initial monitoring image as the background ratio, and obtain the background gray level corresponding to the background pixels; Calculate the variance value of the foreground pixels and the background pixels by using a preset variance formula; Reset multiple classification values, respectively construct corresponding intervals with the set multiple classification values and the first threshold and the second threshold, and execute the step of calculating the variance value. Sort the variance values corresponding to the multiple classification values from large to small to obtain a variance value ranking list; Take the classification value corresponding to the variance value ranked first on the variance value ranking list as the global threshold.

3. The automatic early warning method based on image recognition according to claim 1, characterized in that the binarization processing of the initial monitoring image according to the global threshold to obtain a standard monitoring image includes: setting the pixel points in the initial monitoring image greater than the global threshold to a preset first gray value; setting the pixel points in the initial monitoring image less than or equal to the global threshold to a preset second gray value to obtain a standard monitoring image.

4. The automatic early warning method based on image recognition according to claim 1, characterized in that the screening of multiple skeleton lines in the standard skeleton image includes: detecting multiple lines in the standard skeleton image by using a preset line detection algorithm; deleting the lines with a line length less than a preset line threshold among the multiple lines to obtain multiple skeleton lines.

5. The automatic early warning method based on image recognition according to claim 1, characterized in that the frequency domain conversion and filtering processing of the original monitoring image to obtain an initial monitoring image includes: performing a spatial conversion process on the original monitoring image to obtain a frequency domain image; filtering the frequency domain image by using a preset filtering function to obtain a filtered image; performing a spatial restoration process on the filtered image to obtain an initial monitoring image.

6. The automatic early warning method based on image recognition according to claim 5, characterized in that the performing a spatial conversion process on the original monitoring image to obtain a frequency domain image includes: performing a spatial conversion process on the original monitoring image by using a preset fast Fourier formula to obtain a frequency domain image: Among them, represents the pixel value of the original surveillance image, represents the pixel value of the frequency-domain image, 、 represent the width and height of the original surveillance image, is a fixed parameter in the fast Fourier transform function. x and y respectively represent the x-th row and y-th column in the original surveillance image, 、 respectively represent the -th row and -th column in the frequency-domain image.

7. An automatic early warning device based on image recognition, characterized in that the device includes: a frequency domain conversion module, configured to obtain an original monitoring image, perform a frequency domain conversion and filtering process on the original monitoring image to obtain an initial monitoring image; a binarization module, configured to calculate a global threshold of the initial monitoring image, and perform a binarization process on the initial monitoring image according to the global threshold to obtain a standard monitoring image; a skeleton extraction module, configured to delete pixel points in the standard monitoring image that meet a first preset value or a second preset value according to a preset skeleton extraction condition to obtain a standard skeleton image, where the pixel points of the first preset value or the second preset value are the products of multiple target pixel points that meet a preset first product formula or a second product formula, the multiple target pixel points are respectively the second pixel, the fourth pixel, the sixth pixel, and the eighth pixel, the first preset value or the second preset value is zero, the first product formula is the multiplication of the second pixel, the fourth pixel, and the sixth pixel and the multiplication of the fourth pixel, the sixth pixel, and the eighth pixel, the second product formula is the multiplication of the second pixel, the fourth pixel, and the eighth pixel and the multiplication of the second pixel, the sixth pixel, and the eighth pixel, a total of eight pixel points; A straight-line intersection module, configured to screen multiple skeleton straight lines in the standard skeleton image, perform intersection processing on the multiple skeleton straight lines, and use the intersection points between the straight lines as skeleton key points to obtain multiple skeleton key points; A key area acquisition module, configured to map the multiple skeleton key points to a preset coordinate system, obtain multiple target key areas, and identify the number of the multiple target key areas; An early warning module, configured to perform early warning according to the number of the multiple target key areas and a preset early warning rule.

8. An electronic device, characterized in that, the electronic device includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image recognition-based automatic early warning method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the image recognition-based automatic early warning method according to any one of claims 1 to 6.

Images