Deposits identification method, device and electronic device

Abstract

The present application relates to a method, device and electronic device for identifying deposits, wherein the method for identifying deposits includes: obtaining an image to be detected and a standard image, wherein the standard image does not include deposits; determining the position of each pixel in the image to be detected as a target position, and determining the actual depth information and standard depth information corresponding to each target position; determining the target area in the image to be detected based on the pixels of the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than the deviation value threshold; determining whether there is deposit in the image to be detected based on the target area. Through the present application, the problem of poor accuracy in identifying channel blockage is solved, and the technical effect of identifying the size of deposits through image processing and identifying whether there is blockage based on the size of deposits to improve the accuracy of blockage identification is achieved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance, and in particular to a method, device and electronic device for identifying deposits. Background Art

[0002] With the development of the times and the rapid advancement of urbanization, the population distribution tends to be dense. In high-rise buildings and densely populated areas, smooth passages are essential. When encountering emergencies such as fires, stairs and aisles as public passages are blocked by debris, which can easily lead to congestion, reduce the efficiency of personnel evacuation, and cause stampedes.

[0003] In the prior art, channel blockage identification based on image processing technology is generally performed through background modeling to detect whether the background in the real-time monitoring image has changed. If the background in the real-time monitoring image has changed, a channel blockage alarm is issued. However, background modeling is generally closely related to moving target detection technology. Through moving target detection, it is determined whether the scene in the current real-time monitoring image has changed compared to the unblocked scene in the background model, and then it is determined whether a blockage has occurred. In actual life applications, the appearance of some small objects, such as paper balls, wine bottles and other small debris, although it will not cause channel blockage, will also cause channel blockage alarms due to changes in the background in the monitoring image, resulting in false alarms.

[0004] With regard to the problem of poor accuracy in identifying deposits that cause channel blockage in related technologies, no effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a method, device, electronic device and storage medium for identifying deposits are provided to solve the problem of poor accuracy in identifying channel blockage in the related art.

[0006] In a first aspect, a method for identifying deposits is provided in this embodiment, comprising:

[0007] Acquire an image to be detected and a standard image, wherein the standard image does not include deposits;

[0008] The position of each pixel in the image to be detected is determined as the target position, and the actual depth information and the standard depth information corresponding to each target position are determined; the actual depth information is obtained by mapping the target position in the image to be detected, and the standard depth information is obtained by mapping the corresponding target position in the standard image;

[0009] Determine a target area in the image to be detected based on pixels at the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold;

[0010] Based on the target area, it is determined whether the deposit exists in the image to be detected.

[0011] In one embodiment, the method of respectively determining the position of each pixel in the image to be detected as the target position, and determining the actual depth information and standard depth information corresponding to each target position includes: determining the first plane coordinate information of each pixel in the image to be detected in the image coordinate system of the image to be detected, and determining the depth information obtained by mapping the plane coordinate information in the image to be detected to the reference coordinate system as the actual depth information, and the actual depth information includes actual length information, actual width information, and actual height information; determining the second plane coordinate information of each pixel in the standard image in the image coordinate system of the standard image, and determining the depth information obtained by mapping the second plane coordinate information corresponding to the first plane coordinate information in the standard image to the reference coordinate system as the standard depth information, and the standard depth information includes standard length information, standard width information, and standard height information; determining a first height deviation value based on the actual height information and the standard height information, and using the first height deviation value as the deviation value.

[0012] In one embodiment, the determining of the target area in the image to be detected based on the pixels at the target position where the deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold comprises: obtaining an initial statistical image matrix according to a standard image matrix corresponding to the standard image, the initial statistical image matrix having the same row width and column width as the standard image matrix, and the pixel values ​​at each position in the initial statistical image matrix are all 0; obtaining suspicious position information according to the first plane coordinate information corresponding to the first height deviation value greater than the deviation value threshold in the image to be detected; increasing the pixel value at the same position as the suspicious position information in the initial statistical image matrix by a preset value to obtain a target statistical image; and determining the target area according to the target statistical image.

[0013] In one embodiment, after acquiring the image to be detected and the standard image, the method includes: performing Gaussian modeling based on multiple frames of the standard image to obtain a height background model; comparing the model height information of the pixels of the image to be detected and the pixels at the same position corresponding to the height background model to obtain a second height deviation value.

[0014] In one embodiment, after obtaining the second height deviation value, it also includes: performing motion detection based on multiple frames of the standard image to establish a visible light background model; counting the pixel positions where the second height deviation value is less than a second preset threshold value to determine the position information of the area to be updated; obtaining the area to be updated according to the position information of the area to be updated; performing feature comparison between the image of the area to be updated in the image to be detected and the image of the area to be updated in the visible light background model to obtain feature matching parameters; if the feature matching parameters are greater than a third preset threshold value, replacing the actual height information and corresponding pixel values ​​of the images of the area to be updated in the visible light background model and the height background model according to the position information of the area to be updated in the image to be detected and the corresponding pixel values.

[0015] In one embodiment, determining whether the deposit exists in the image to be detected based on the target area includes: determining actual size information of the suspicious object in the image to be detected based on the target position of each pixel corresponding to the target area in the image to be detected and the actual depth information; and determining whether the suspicious object is a deposit based on the actual size information.

[0016] In one embodiment, the actual size information includes length information, height information and width information, and determining whether the suspicious object is an accumulation based on the actual size information includes: comparing the length information with a length threshold to obtain a first comparison result; comparing the width information with a width threshold to obtain a second comparison result; comparing the height information with a height threshold to obtain a third comparison result; and determining whether the suspicious object in the image to be detected is an accumulation based on the first comparison result, the second comparison result and the third comparison result, and if the suspicious object is an accumulation, determining the blockage level of the accumulation.

[0017] In one embodiment, the image to be detected and the standard image are collected by a binocular camera.

[0018] In a second aspect, a device for identifying deposits is provided in this embodiment, comprising:

[0019] A first acquisition module is used to acquire an image to be detected and a standard image, wherein the standard image does not include deposits;

[0020] A second acquisition module is used to determine the position of each pixel in the image to be detected as the target position, and determine the actual depth information and standard depth information corresponding to each target position; the actual depth information is obtained by mapping the target position in the image to be detected, and the standard depth information is obtained by mapping the corresponding target position in the standard image;

[0021] A calculation module, which determines a target area in the image to be detected by using pixels at the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold;

[0022] A judgment module is used to determine whether the deposit exists in the image to be detected based on the target area.

[0023] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for identifying deposits described in the first aspect when executing the computer program.

[0024] Compared with the related art, the blockage identification method provided in the present embodiment obtains an image to be detected and a standard image, wherein the standard image does not include deposits; the position of each pixel in the image to be detected is determined as the target position, and the actual depth information and the standard depth information corresponding to each target position are determined; the actual depth information is obtained by mapping the target position in the image to be detected, and the standard depth information is obtained by mapping the corresponding target position in the standard image; based on the pixels of the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than the deviation value threshold, the target area in the image to be detected is determined; based on the target area, it is determined whether the deposits exist in the image to be detected, thereby solving the problem of poor accuracy in identifying deposits and achieving the technical effect of accurately identifying deposits through image processing methods.

[0025] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0027] Figure 1 is a hardware structure block diagram of a terminal of the deposit identification method of this embodiment;

[0028] Figure 2 is a flow chart of the deposit identification method of this embodiment;

[0029] Figure 3 is a flow chart of the method for identifying deposits in the preferred embodiment;

[0030] Figure 4 It is a structural block diagram of the deposited object identification device of this embodiment. DETAILED DESCRIPTION

[0031] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0032] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0033] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 4 is a hardware structure diagram of a terminal of the method for identifying deposits in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0034] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the deposit identification method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0036] For firefighting, personnel evacuation is an eternal topic, especially personnel evacuation in case of fire. Fire passages are special passages for firefighters to extinguish fires and rescue and evacuate trapped people in fires. They are also escape passages for trapped people in fires. However, in daily life, due to the weak firefighting awareness of some people, they pile up personal belongings in public passages at will. When the number of piled items is large or the items themselves are large in size, it is very easy to cause passage blockage. Not only does it bring inconvenience to the passage of people in normal times, but when encountering situations such as fires that require emergency evacuation of people, the accumulation of objects will greatly reduce the efficiency of personnel evacuation, and even cause significant casualties and property losses.

[0037] In this embodiment, a method for identifying deposits is provided. Figure 2 is a flow chart of the method for identifying deposits in this embodiment, such as Figure 2 As shown, the process includes the following steps:

[0038] Step S201, obtaining an image to be detected and a standard image.

[0039] Specifically, the image to be detected is a scene image acquired by a camera device and needs to be identified for accumulation. For example, images of stairways, doorways, etc. Accumulation refers to one or more items whose volume or size exceeds a preset range and causes space blockage. For example, bicycles in the corridor, piled up large packages, etc., are all accumulations. The determination of accumulations is generally related to specific detection scenes. These detection scenes generally include fire escapes, stairways, warehouse entrances and exits, and other relatively narrow locations with large flow of people or goods. When an object may cause blockage in the current scene, the object can be regarded as accumulation. Accumulation is not included in the standard image. The standard image is a pre-collected, unobstructed scene image without accumulation, and the standard image does not include accumulation. The image to be detected is an image of the current scene to be detected collected in real time.

[0040] Step S202 : determining the position of each pixel in the image to be detected as a target position, and determining the actual depth information and the standard depth information corresponding to each target position.

[0041] Specifically, the image coordinate systems commonly used in the field of photography include image coordinate systems, imaging plane coordinate systems, camera coordinate systems, and world coordinate systems. The image coordinate system refers to the digital image captured by the camera can be stored as an array in the computer, and the value of each element in the array is the brightness of the image point, that is, the grayscale. The array is usually represented by a matrix. A rectangular coordinate system is defined on the image, and the coordinates (i, j) of each pixel are the number of columns and rows of the pixel in the array, respectively, so (i, j) is the image coordinate system coordinate in pixels. The coordinates (i, j) are used as the target position corresponding to the pixel. The world coordinate system refers to selecting a reference coordinate system in the environment to describe the position of the camera and the object, and this coordinate system is called the world coordinate system. The relationship between the camera coordinate system and the world coordinate system can be described by a rotation matrix and a translation vector. In other words, the three-dimensional coordinate information obtained by mapping the target position (i, j) of the pixel to the world coordinate system is the depth information.

[0042] The actual depth information is obtained by mapping the target position in the image to be detected. The standard depth information is obtained by mapping the corresponding target position in the standard image. For example, the target position corresponding to a pixel in the image to be detected is (1, 1), and the depth information of the target position mapped to the world coordinate system is (0, 0, 30). Then the actual depth information is (0, 0, 30); the corresponding target position (1, 1) in the standard image is mapped to the depth information of the world coordinate system (0, 0, 20), then the standard depth information is (0, 0, 20). If the coordinates of the world coordinate system are in centimeters, it means that the height of the position when there is no accumulation is 20 centimeters, and the actual height of the current position detected is 30 centimeters. Step S203, based on the pixels of the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than the deviation value threshold, determine the target area in the image to be detected.

[0043] Specifically, the actual depth information is compared with the standard depth information to obtain the deviation value between the actual depth information and the standard depth information. For example, if the world coordinate system is in centimeters, the actual depth information corresponding to a certain target position (1, 1) obtained from the image to be detected is (0, 0, 30); the standard depth information corresponding to the corresponding target position (1, 1) obtained from the standard image is (0, 0, 20); by comparing the actual depth information with the standard depth information, it can be seen that the deviation value between the actual depth information and the standard depth information on the Z axis is 10, that is, the height of the position when there is no accumulation is 20 centimeters, and the actual height of the current position detected is 30 centimeters. Compared with the standard unblocked scene, there are extra items at the current position, and the height of the item at this position is 10 centimeters. The user can set the deviation value threshold according to actual needs. The purpose of setting the deviation value threshold is to screen the position with deviations to avoid the situation where the error between the image to be detected and the standard image due to visible light causes false detection, or because the deviation value is small, it will not cause scene blockage, avoiding false detection. The target position and the corresponding pixel whose deviation value is greater than the deviation value threshold are counted. The target area in the image to be detected, that is, the area where the suspicious deposits exist, is determined based on the target position and the pixels.

[0044] Step S204: determining whether the deposit exists in the image to be detected based on the target area.

[0045] Specifically, based on the recognition algorithm or based on the size of the target area in the image to be detected, it is determined whether there is an accumulation in the image to be detected. For example, the accumulation in the image is identified by a pre-trained neural network model or a machine learning model. For another example, a regional area threshold in the image is set. When the area of ​​the target area in the image to be detected is greater than the regional area threshold, it is determined that there is an accumulation in the image to be detected. Preferably, the object size corresponding to the target area is calculated based on the actual depth information and standard depth information corresponding to the pixels in the target area. If the size of the object is greater than the size threshold, it is determined that there is an accumulation in the image to be detected.

[0046] Through the above steps, the present application obtains the three-dimensional spatial information of the actual scene corresponding to each pixel in the image, compares the actual depth information corresponding to the pixel of the current image to be detected with the standard depth information corresponding to the pixel at the same position in the standard image, obtains the deviation value, counts the positions of the corresponding pixels that meet the height difference, obtains the suspected blockage area based on the position, obtains the size information of the corresponding deposit based on the blockage area, and obtains the blockage identification result based on the comparison result of the deposit size with the preset threshold value. This solves the problem that the prior art uses the background state model to perform blockage identification, issues an alarm when a background change is detected, and is easily interfered by small debris, resulting in misjudgment, and achieves the technical effect of accurate blockage identification.

[0047] In one embodiment, the method of respectively determining the position of each pixel in the image to be detected as the target position, and determining the actual depth information and standard depth information corresponding to each target position includes: determining the first plane coordinate information of each pixel in the image to be detected in the image coordinate system of the image to be detected, and determining the depth information obtained by mapping the plane coordinate information in the image to be detected to the reference coordinate system as the actual depth information, and the actual depth information includes actual length information, actual width information, and actual height information; determining the second plane coordinate information of each pixel in the standard image in the image coordinate system of the standard image, and determining the depth information obtained by mapping the second plane coordinate information corresponding to the first plane coordinate information in the standard image to the reference coordinate system as the standard depth information, and the standard depth information includes standard length information, standard width information, and standard height information; determining a first height deviation value based on the actual height information and the standard height information, and using the first height deviation value as the deviation value.

[0048] Specifically, the image to be detected and the standard image are obtained by a binocular camera. The binocular camera is set up at a certain angle to obtain the image to be detected of the current scene. The standard image can be a pre-acquired, continuous multi-frame unblocked image based on the current scene. The position of each pixel in the image to be detected can be represented by the pixel position coordinate (i, j), that is, the first plane coordinate information, wherein i indicates that the pixel is located in the i-th row of the image matrix to be detected corresponding to the image to be detected, and j indicates that the pixel is located in the j-th column of the image matrix to be detected corresponding to the image to be detected. Since the image is collected by the binocular camera, by calibrating the internal and external parameters, the first plane coordinate information of the pixel in the image to be detected can be mapped to the actual position information in the world coordinate system, that is, the actual depth information. The position coordinates (i, j) of the pixel correspond to the actual depth information (x, y, z) of the corresponding position of the pixel in the scene to be detected. Similarly, the second plane coordinate information (u, v) of the standard image pixel of the standard detection scene and the corresponding standard depth information (m, n, k) are obtained. If the first position coordinate (i0, j0) of the pixel to be detected is the same as the second position coordinate (u0, v0) of the standard image pixel, the actual height information z0 in the actual depth information corresponding to the pixel to be detected is compared with the standard height information k0 in the standard depth information to obtain the height deviation value.

[0049] In one embodiment, the determining of the target area in the image to be detected based on the pixels at the target position where the deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold comprises: obtaining an initial statistical image matrix according to a standard image matrix corresponding to the standard image, the initial statistical image matrix having the same row width and column width as the standard image matrix, and the pixel values ​​at each position in the initial statistical image matrix are all 0; obtaining suspicious position information according to the first plane coordinate information corresponding to the first height deviation value greater than the deviation value threshold in the image to be detected; increasing the pixel value at the same position as the suspicious position information in the initial statistical image matrix by a preset value to obtain a target statistical image; and determining the target area according to the target statistical image.

[0050] Specifically, digital image data can be represented by a matrix, so the digital image can be analyzed and processed using matrix theory and matrix algorithms. Since digital images can be represented in the form of a matrix, two-dimensional arrays are usually used to store image data in computer digital image processing programs. The disparity map refers to the position deviation of pixels of the same scene imaged under two cameras. Establish matrices X, Y, and Z of the same size as the disparity map of the binocular camera, respectively containing actual length information x, actual width information y, and actual height information z corresponding to (i, j). The initial statistical image matrix is ​​obtained according to the standard image matrix. Since the initial statistical image matrix has the same row width and column width as the standard image matrix, after obtaining the suspicious position information, the pixel values ​​at the same position in the initial statistical image matrix are added with preset values. For example, if the suspicious position information is (1, 2) and (2, 2), then 1 is added to the pixel values ​​with position information (1, 2) and (2, 2) in the standard image matrix to obtain the target statistical image matrix, and the target statistical image is obtained according to the target statistical image matrix, and then the suspicious blocking area, that is, the target area, is obtained.

[0051] In one embodiment, multiple frames of images to be detected are compared with standard images respectively. After each comparison to obtain suspicious position information, a preset value is added to the pixel value at the same position as the suspicious position information in the initial statistical image matrix, that is, the initial statistical image matrix is ​​accumulated multiple times to obtain a target statistical image matrix. The suspicious blockage area is obtained according to the target statistical image matrix. Through multiple accumulations, the accuracy of the target statistical image can be improved, so that the obtained suspicious blockage area is more accurate.

[0052] In one embodiment, obtaining the suspected congestion area according to the target statistical image includes: binarizing the image based on the target statistical image to obtain a binarized statistical image; obtaining the suspected congestion area according to the binarized statistical image. Specifically, an initial binary frequency matrix is ​​obtained, and the initial binary frequency matrix is ​​the same as the initial statistical image matrix. A preset statistical threshold is obtained, and the pixel values ​​at each position of the target statistical image matrix are compared with the preset statistical threshold to obtain the position information of the pixel values ​​greater than the preset statistical threshold. According to the position information of the pixel values ​​greater than the preset statistical threshold, the pixel values ​​at the same position in the initial binary frequency matrix are set to 1, and the remaining positions are set to 0 to obtain a binarized statistical image. The suspected congestion area, i.e., the target area, is obtained according to the binarized statistical image.

[0053] In one embodiment, after acquiring the image to be detected and the standard image, the method includes: performing Gaussian modeling based on multiple frames of the standard image to obtain a height background model; comparing the model height information of the pixels of the image to be detected and the pixels at the same position corresponding to the height background model to obtain a second height deviation value.

[0054] In one embodiment, after obtaining the second height deviation value, it also includes: performing motion detection based on multiple frames of the standard image to establish a visible light background model; counting the pixel positions where the second height deviation value is less than a second preset threshold value to determine the position information of the area to be updated; obtaining the area to be updated according to the position information of the area to be updated; performing feature comparison between the image of the area to be updated in the image to be detected and the image of the area to be updated in the visible light background model to obtain feature matching parameters; if the feature matching parameters are greater than a third preset threshold value, replacing the actual height information and corresponding pixel values ​​of the images of the area to be updated in the visible light background model and the height background model according to the position information of the area to be updated in the image to be detected and the corresponding pixel values.

[0055] Specifically, Gaussian modeling is used to model the motion background of the left image in the visible light binocular image to obtain a visible light background model. The pixel position whose height difference is less than the second preset threshold is used as the area to be updated. Preferably, the second preset threshold is 5 cm. Image comparison is performed on the current visible light image area and the visible light background model area, image features in the area are extracted, and feature comparison is performed. If the feature matching ratio is greater than 80%, it is considered that the height and visible light pixel changes in the area are small. At this time, the position information of the area to be updated in the image to be detected and the corresponding pixel value are used to update the height model and the height information and pixel value of the corresponding area in the visible light background model. Through the visible light background model, interference caused by light, water stains, etc. is avoided, and the accuracy of the height background model is improved.

[0056] In one embodiment, determining whether the deposit exists in the image to be detected based on the target area includes: determining actual size information of the suspicious object in the image to be detected based on the target position of each pixel corresponding to the target area in the image to be detected and the actual depth information; and determining whether the suspicious object is a deposit based on the actual size information.

[0057] In one embodiment, the actual size information includes length information, height information and width information, and determining whether the suspicious object is an accumulation based on the actual size information includes: comparing the length information with a length threshold to obtain a first comparison result; comparing the width information with a width threshold to obtain a second comparison result; comparing the height information with a height threshold to obtain a third comparison result; and determining whether the suspicious object in the image to be detected is an accumulation based on the first comparison result, the second comparison result and the third comparison result, and if the suspicious object is an accumulation, determining the blockage level of the accumulation.

[0058] Specifically, the first comparison result includes that the length of the suspicious item is greater than or equal to the length threshold and the length of the suspicious item is less than the length threshold; the second comparison result includes that the width of the suspicious item is greater than or equal to the width threshold and the width of the suspicious item is less than the width threshold; the third comparison result includes that the height of the suspicious item is greater than or equal to the height threshold and the height of the suspicious item is less than the height threshold. The user can set the judgment condition according to actual needs. When one or more of the first comparison result, the second comparison result and the third comparison result is that the actual size information of the suspicious item is greater than the corresponding threshold, the suspicious item is determined to be an accumulation. Preferably, when the length of the suspicious item is greater than or equal to the length threshold, the width of the suspicious item is greater than or equal to the width threshold and the height of the suspicious item is greater than or equal to the height threshold, the suspicious item is determined to be an accumulation. In addition, the user can set a multi-level threshold, which corresponds to the blocking level. For example: set the first height threshold to 20 cm, the second height threshold to 40 cm, and the third height threshold to 60 cm. The corresponding blocking levels are light blocking accumulation, moderate blocking accumulation and high blocking accumulation.

[0059] In one embodiment, the image to be detected and the standard image are collected by a binocular camera.

[0060] The present embodiment is described and illustrated below through preferred embodiments.

[0061] Figure 3 FIG. 4 is a flow chart of the method for identifying deposits in the preferred embodiment. Figure 3 As shown in the figure, the binocular camera is set up at 45 degrees, and the camera internal and external parameters are calibrated. The 3D information (x, y, z) of each pixel in the image is obtained through the binocular camera, and the coordinates (i, j) of each pixel in the image correspond to the 3D information. The upper left corner of the image corresponds to the origin in the world coordinate system, x is the length of the corresponding coordinate origin; y is the width of the corresponding coordinate origin; z is the height of the corresponding coordinate origin. Establish matrices X, Y, and Z of the same size as the disparity map, and contain the x, y, and z information corresponding to (i, j). Obtain the initial statistical image matrix and the initial binary frequency matrix, and perform height modeling on the height matrix.

[0062] Acquire multiple frames of unblocked images, and use Gaussian background modeling to obtain a height background Gaussian model; similarly, use Gaussian modeling to model the motion background of the left image in the visible light binocular image to obtain a visible light background model.

[0063] After modeling, multiple frames of images to be detected are obtained. In each comparison process, the height matrix corresponding to the pixels of the image to be detected is compared with the height model corresponding to the unblocked graphics, and the height difference at each position is recorded; a threshold A is set, and the preferred threshold A is 40 cm. If the height difference is greater than the threshold A, the position corresponding to the height difference in the initial statistical image matrix is ​​added by 1, and then the target statistical image matrix is ​​obtained. Based on the obtained target statistical image matrix, processing is performed, and for positions where the pixel value is greater than the second threshold B, the corresponding position in the initial binary frequency matrix is ​​set to 1, otherwise it is set to 0. A binary statistical image is obtained based on the set initial binary frequency matrix. Preferably, the second threshold B is 100.

[0064] According to the binary statistical image, the outer contour detection is performed to obtain the suspected congestion area; the suspected congestion area is (Zx1, Zy1, Zx2, Zy2), where (Zx1, Zy1) represents the image pixel coordinates of the upper left corner vertex of the external detection box of the suspected congestion area; (Zx2, Zy2) represents the image pixel coordinates of the lower right corner vertex of the external detection box of the suspected congestion area. According to the suspected congestion area, the corresponding three-dimensional coordinate moments X, Y, and Z of the suspected congestion area in the image to be detected are obtained. The matrix X, Y, and Z are median filtered 5*5 to avoid abnormal values. The length, width, and height of the deposit are calculated. The pre-configured size thresholds include: the length threshold is 40 cm, the width threshold is 50 cm, and the height threshold is 60 cm. If the length, width, and height of the deposit are all greater than the corresponding thresholds, the channel is judged to be blocked and an alarm is issued. The alarm method can be a voice prompt or a light prompt, etc. The present invention does not specifically limit the alarm rules.

[0065] It should be emphasized that after obtaining the height difference, a second preset threshold M is obtained, wherein the second preset threshold M is less than the threshold A, and preferably, the second preset threshold M is 5 cm. If the height difference is less than the second preset threshold M, the area corresponding to the height difference that satisfies this condition is taken as the satisfying area N. Based on satisfying area N, image comparison is performed on the current visible light image area and the visible light background model area, image features in the area are extracted, and feature comparison is performed. If the feature matching ratio is greater than the third feature threshold, for example, the third feature threshold is 80%, it is considered that the height and visible light pixel changes in the area are small, and the height information and pixel information of the corresponding area in the height model and the visible light background model are updated according to the pixel height information and pixel value in the satisfying area N in the image to be detected.

[0066] Through the above steps, the present application uses a binocular camera to obtain the 3D information of the corresponding target in the image, models the height information to obtain the height background model, and models the motion background of the visible light image; obtains the suspicious position information by comparing the current height and the modeling height; counts the frequency of occurrence of pixels with height difference values ​​to obtain the height difference frequency map, that is, the target statistical image; and binarizes the target statistical image, and obtains the suspicious blocking area through the clustering algorithm. Based on the suspicious blocking contour, the length, width and height information of the suspicious accumulation object are obtained in combination with the 3D information, and the target that meets the blocking rule is alarmed. It can effectively filter the interference of non-blocking objects and reduce the false alarm rate of blocking identification. In addition, the present application also updates the height model and the visible light background model based on the real-time acquired image to be detected, by obtaining the area with a small difference between the current height and the modeling height; compares the current visible light area and the visible light background area, and calculates the corresponding area difference through feature comparison. If the difference is less than the set rule, the height model and the visible light background model corresponding area data are updated. Interference caused by light and water stains is avoided. Moreover, compared with the prior art method of identifying blocked objects based on deep learning models, the method for identifying deposits proposed in the present application identifies blockages based on the size of the deposits, which is more applicable and can better meet the needs of real scenarios.

[0067] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0068] In this embodiment, a device for identifying deposits is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit" and the like used below can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceivable.

[0069] Figure 4 : is a block diagram of the structure of the deposit identification device of this embodiment, such as Figure 4 As shown, the device comprises:

[0070] A first acquisition module 10 is used to acquire an image to be detected and a standard image, wherein the standard image does not include deposits;

[0071] A second acquisition module 20 is used to determine the position of each pixel in the image to be detected as the target position, and determine the actual depth information and standard depth information corresponding to each target position; the actual depth information is obtained by mapping the target position in the image to be detected, and the standard depth information is obtained by mapping the corresponding target position in the standard image;

[0072] The calculation module 30 determines the target area in the image to be detected by using the pixels at the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than the deviation value threshold;

[0073] The judgment module 40 is used to determine whether the deposit exists in the image to be detected based on the target area.

[0074] The second acquisition module 20 is also used to determine the first plane coordinate information of each pixel in the image to be detected in the image coordinate system of the image to be detected, and determine the depth information obtained by mapping the plane coordinate information in the image to be detected to the reference coordinate system as the actual depth information, and the actual depth information includes actual length information, actual width information and actual height information; determine the second plane coordinate information of each pixel in the standard image in the image coordinate system of the standard image, and determine the depth information obtained by mapping the second plane coordinate information corresponding to the first plane coordinate information in the standard image to the reference coordinate system as the standard depth information, and the standard depth information includes standard length information, standard width information and standard height information; determine a first height deviation value based on the actual height information and the standard height information, and use the first height deviation value as the deviation value.

[0075] The calculation module 30 is further used to obtain an initial statistical image matrix according to the standard image matrix corresponding to the standard image, wherein the initial statistical image matrix has the same row width and column width as the standard image matrix, and the pixel values ​​at each position in the initial statistical image matrix are all 0; obtain suspicious position information according to the first plane coordinate information corresponding to the first height deviation value greater than the deviation value threshold in the image to be detected; increase the pixel value in the initial statistical image matrix at the same position as the suspicious position information by a preset value to obtain a target statistical image; and determine the target area according to the target statistical image.

[0076] The second acquisition module 20 is also used to perform Gaussian modeling based on multiple frames of the standard image to obtain a height background model; compare the model height information of the pixels of the image to be detected with the pixels at the same position corresponding to the height background model to obtain a second height deviation value.

[0077] The calculation module 30 is also used to perform motion detection based on multiple frames of the standard image to establish a visible light background model; count the pixel positions whose second height deviation value is less than the second preset threshold value to determine the position information of the area to be updated; obtain the area to be updated according to the position information of the area to be updated; perform feature comparison on the image of the area to be updated in the image to be detected and the image of the area to be updated in the visible light background model to obtain feature matching parameters; if the feature matching parameters are greater than a third preset threshold value, replace the actual height information and corresponding pixel values ​​of the images of the area to be updated in the visible light background model and the height background model according to the position information of the area to be updated in the image to be detected and the corresponding pixel values.

[0078] The judgment module 40 is further used to determine the actual size information of the suspicious object in the image to be detected based on the target position of each pixel corresponding to the target area in the image to be detected and the actual depth information; based on the actual size information, determine whether the suspicious object is a pile.

[0079] The judgment module 40 is further used to compare the length information with the length threshold to obtain a first comparison result; compare the width information with the width threshold to obtain a second comparison result; compare the height information with the height threshold to obtain a third comparison result; and determine whether the suspicious object in the image to be detected is an accumulation according to the first comparison result, the second comparison result and the third comparison result, and if the suspicious object is an accumulation, determine the blockage level of the accumulation.

[0080] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0081] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0082] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0083] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0084] S1, obtain the image to be detected and the standard image.

[0085] S2, respectively determining the position of each pixel in the image to be detected as a target position, and determining the actual depth information and the standard depth information corresponding to each target position.

[0086] S3, determining a target area in the image to be detected based on pixels at the target position where a deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold.

[0087] S4, based on the target area, determining whether the deposit exists in the image to be detected.

[0088] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0089] In addition, in combination with the congestion identification method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the congestion identification method. The storage medium stores a computer program; when the computer program is executed by a processor, any congestion identification method in the above embodiments is implemented.

[0090] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0091] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0092] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0093] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for identifying deposits, characterized in that: include: Acquire an image to be detected and a standard image, wherein the standard image does not include deposits; After acquiring the image to be detected and the standard image, the method includes: performing Gaussian modeling based on multiple frames of the standard image to obtain a height background model; comparing the model height information of the pixels of the image to be detected with the pixels at the same position corresponding to the height background model to obtain a second height deviation value; The position of each pixel in the image to be detected is determined as the target position respectively, and the actual depth information and standard depth information corresponding to each target position are determined; the actual depth information is obtained by mapping the target position in the image to be detected; the actual depth information includes actual height information; the standard depth information is obtained by mapping the corresponding target position in the standard image; the standard depth information includes standard height information; after obtaining the second height deviation value, it also includes: performing motion detection based on multiple frames of the standard image to establish a visible light background model; counting the pixel positions whose second height deviation value is less than the second preset threshold value to determine the position information of the area to be updated; obtaining the area to be updated according to the position information of the area to be updated; performing feature comparison between the image of the area to be updated in the image to be detected and the image of the area to be updated in the visible light background model to obtain feature matching parameters; if the feature matching parameters are greater than a third preset threshold, then replacing the actual height information and corresponding pixel values ​​of the image of the area to be updated in the visible light background model and the height background model according to the position information and corresponding pixel values ​​of the area to be updated in the image to be detected; Determine a target area in the image to be detected based on pixels at the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold; Based on the target area, it is determined whether the deposit exists in the image to be detected.

2. The method for identifying deposits according to claim 1, characterized in that: The step of respectively determining the position of each pixel in the image to be detected as the target position, and determining the actual depth information and the standard depth information corresponding to each target position includes: Determine first plane coordinate information of each pixel in the image to be detected in the image coordinate system of the image to be detected, and determine depth information obtained by mapping the plane coordinate information in the image to be detected to a reference coordinate system as the actual depth information, wherein the actual depth information includes actual length information, actual width information, and actual height information; Determine second plane coordinate information of each pixel in the standard image in the image coordinate system of the standard image, and determine depth information obtained by mapping the second plane coordinate information corresponding to the first plane coordinate information in the standard image to the reference coordinate system as the standard depth information, wherein the standard depth information includes standard length information, standard width information, and standard height information; A first height deviation value is determined based on the actual height information and the standard height information, and the first height deviation value is used as the deviation value.

3. The method for identifying deposits according to claim 2, characterized in that: The determining of the target area in the image to be detected based on the pixel at the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than the deviation value threshold comprises: Acquire an initial statistical image matrix according to a standard image matrix corresponding to the standard image, wherein the initial statistical image matrix has the same row width and column width as the standard image matrix, and the pixel value at each position in the initial statistical image matrix is ​​0; Obtaining suspicious position information according to the first plane coordinate information corresponding to the first height deviation value greater than the deviation value threshold in the image to be detected; The pixel values ​​in the initial statistical image matrix having the same position as the suspicious position information are increased by a preset value to obtain a target statistical image; The target area is determined according to the target statistical image.

4. The method for identifying deposits according to claim 1, characterized in that: The determining, based on the target area, whether the deposit exists in the image to be detected includes: Determining actual size information of the suspicious object in the image to be detected based on the target position of each pixel corresponding to the target area in the image to be detected and the actual depth information; Based on the actual size information, it is determined whether the suspicious object is a deposit.

5. The method for identifying deposits according to claim 4, characterized in that: The actual size information includes length information, height information and width information. The determining whether the suspicious object is a piled object based on the actual size information includes: Comparing the length information with a length threshold to obtain a first comparison result; Comparing the width information with a width threshold to obtain a second comparison result; Comparing the height information with the height threshold to obtain a third comparison result; According to the first comparison result, the second comparison result and the third comparison result, it is determined whether the suspicious object in the image to be detected is a deposit; if the suspicious object is a deposit, the blockage level of the deposit is determined.

6. The method for identifying deposits according to claim 1, characterized in that: The image to be detected and the standard image are collected by a binocular camera.

7. A deposit identification device, characterized in that: include: A first acquisition module is used to acquire an image to be detected and a standard image, wherein the standard image does not include deposits; After acquiring the image to be detected and the standard image, the method includes: performing Gaussian modeling based on multiple frames of the standard image to obtain a height background model; comparing the model height information of the pixels of the image to be detected with the pixels at the same position corresponding to the height background model to obtain a second height deviation value; The second acquisition module is used to determine the position of each pixel in the image to be detected as the target position, and determine the actual depth information and standard depth information corresponding to each target position; the actual depth information is obtained by mapping the target position in the image to be detected; the actual depth information includes actual height information; the standard depth information is obtained by mapping the corresponding target position in the standard image; the standard depth information includes standard height information; after obtaining the second height deviation value, it also includes: performing motion detection based on multiple frames of the standard image to establish a visible light background model; counting the pixel positions whose second height deviation value is less than the second preset threshold value to determine the position information of the area to be updated; obtaining the area to be updated according to the position information of the area to be updated; performing feature comparison between the image of the area to be updated in the image to be detected and the image of the area to be updated in the visible light background model to obtain feature matching parameters; if the feature matching parameters are greater than a third preset threshold, then replacing the actual height information and corresponding pixel values ​​of the image of the area to be updated in the visible light background model and the height background model according to the position information and corresponding pixel values ​​of the area to be updated in the image to be detected; A calculation module, used to determine a target area in the image to be detected based on pixels at the target position whose deviation value between the actual depth information and the standard depth information in the image to be detected is greater than a deviation value threshold; A judgment module is used to determine whether the deposit exists in the image to be detected based on the target area.

8. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the deposit identification method according to any one of claims 1 to 6.

Images