A recognition method for abnormal behaviors in a sorting center based on video detection technology

Abstract

The present invention discloses an automatic recognition method for abnormal behaviors in a sorting center based on video detection technology. In view of the monotonicity of the indoor light source environment and foreground types in the sorting center environment, a more targeted video detection method is designed. At the same time, targeted detections are carried out on the main detection objects in the sorting center, namely the express packages and the activities of sorting personnel. According to the characteristics of abnormal behaviors in the sorting center, a series of methods for detecting abnormal situations are proposed.

Description

Technical Field

[0001] The present invention relates to a method for identifying abnormal behaviors in a sorting center based on video detection technology, belonging to the fields of logistics management and video detection technology. Background Art

[0002] The throughput of the express sorting center is extremely large, and the number of its staff is small. It is often difficult to complete the inspection of abnormal express sorting in addition to sorting and forwarding the express. In most cases, it is necessary for users to feedback or conduct a data census to discover the retention problems caused by abnormal sorting, which seriously affects the user experience.

[0003] By detecting the continuous pictures collected by the camera, the motion conditions of the key objects therein can be obtained. Combining with the specific working scene where the camera is located, a set of targeted video detection systems can be designed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to specifically propose a variety of video detection methods applicable to the specific environment of the sorting center in view of the pain points existing in the actual production process of the logistics sorting center in the prior art, and make professional improvements to the filtering, dehazing, background extraction and motion recognition required in video detection.

[0005] To achieve the above object, the technical solution adopted by the present invention is: an automatic recognition method for the operation specifications of a sorting center based on video detection technology, which method includes two parts: scene environment parameter adaptation and real-time video detection. Among them, for scene environment parameter adaptation, before the real-time video detection officially starts, it is necessary to pre-load some video frames to analyze the scene environment and set some parameters in the video detection process, including the environmental atmospheric light matrix based on point light sources, the window size σ and enhancement coefficient k of the adaptive detail enhancement filter, and the global background threshold matrix based on scene smoothness.

[0006] In the images collected by the camera, there are many noise points. These noise points do not affect human eye recognition, but will cause great interference to computer intelligent image processing. Common dehazing means such as those based on the priori dark channel can process the image by calculating the atmospheric light constant and substituting it into the formula to achieve the effect of dehazing and noise reduction.

[0007] Under normal circumstances, sunlight can be regarded as parallel light. Therefore, within the area covered by the camera, the atmospheric light conditions in each area are the same. However, in a relatively enclosed indoor environment, the illumination is often provided by lamps. Due to the influence of the quality and position of the lamps, the indoor illumination in the sorting center can be regarded as a composite light environment of multiple point light sources. In this case, a single atmospheric light constant is no longer sufficient to describe the illumination environment of the entire scene. Therefore, it is necessary to calculate the global illumination matrix to replace the original atmospheric light constant.

[0008] The calculation principle of the global illumination matrix is to adjust the illumination matrix within the entire image according to the influence of the local brightness intensity characteristics of the dark channel map on the surrounding light environment.

[0009] First, the system performs type conversion on the input image of size m×n, and calculates its dark channel map. The dark channel map Dark i,j = min(J C (i,j)), 0 ≤ i < m, 0 ≤ j < n, C ∈ {R, G, B}, where J C (i,j) refers to the value of the pixel at coordinate (i,j) in the image J on the C-channel component.

[0010] Secondly, set the step size step as and the window size as Starting from the upper left corner of the image, translate step by step to the right, and obtain 15 image patches. And translate the columns in the same way, obtain 15×15 image patches from the image, and calculate the brightness characteristics of each image patch Divide p,q = max(Dark i,j ),(i,j) ∈ Ω(p,q), 0 ≤ p < 15, 0 ≤ q < 15, where Ω(p,q) is the image patch in the p-th row and q-th column divided, and (i,j) ∈ Ω(p,q) refers to the coordinates of the pixels belonging to this image patch.

[0011] Then, combine the image patches obtained in the previous step. Set the new window as (3,3), perform non-overlapping translation segmentation on the matrix Divide i,j to obtain a set of 5×5 image groups, and each image group includes 9 image patches. And calculate the eigenvalue of each image group to obtain the image group feature matrix as Among them, Divide t,s refers to the image patch included in the image group in the t-th row and s-th column, refers to the i-th image patch from left to right and top to bottom in this image group.

[0012] Through this series of data cleaning and filtering, three-level features of image brightness are obtained. Immediately afterwards, starting from the image group features, we gradually reduce the brightness features level by level.

[0013] First, correct Divide according to the situation of Var. When the eigenvalue of the image group is greater than the average eigenvalue, use the maximum value of the features of each image block in the image group to replace all the values within the group. When the average value of the image group is less than the average eigenvalue, use the average feature value of each image block in the image group to replace all the values within the group. The correction formula is:

[0014]

[0015] After completing the correction of the image block features, since most pixels are covered by multiple image blocks, we take the maximum value of the features of the image blocks covering the pixels as the atmospheric light constant at this point, denoted as A i,j =max(Divide)where(i,j)in Divide. A is the final image illumination matrix.

[0016] Due to the complex environment of the express sorting center, the requirements for the recognition of fine behaviors and objects are also increased accordingly. Due to the problem of edge passivation in the directly obtained images, there are often certain problems in the detail performance. Although it is more in line with the naked-eye vision, it is not conducive to machine processing. Therefore, we need to further process the images collected by the camera to strengthen their detail features.

[0017] Here, we adopt adaptive high-enhancement filtering. In this process, first, we need to determine the window size for filtering in this scene. We first give a smaller window, and then continuously expand the window size while calculating a series of feature parameters.

[0018] First, for the input image J, there is its dark channel image B x =0.59×J R (x)+0.3×J G (x)+ 0.11×J B (x). Among them, J C (x) refers to the value of the pixel point x of the image J on the C channel component.

[0019] Secondly, we set the initial window size to 3×3 and perform non-overlapping translation segmentation on the entire image with this size, dividing the entire image into windows and obtaining the window set U. There is Feature k =Quantile 0.99 (B x )-avg(B x), x ∈ Ω(k), where Ω(k) is the k-th partition window of B, Quantile p (U) refers to the p-th quantile of set U, and avg(U) refers to the average of set U.

[0020] When more than 75% of the Feature k <Quantile 0.75 (B x ) - Quantile 0.25 (B x ) 1.5 , expand the window to 1.3 times and continuously calculate its eigenvalue until the maximum window size σ×σ that meets the conditions is found.

[0021] Then, according to the eigenvalue of the determined maximum window, find the enhancement coefficient

[0022]

[0023] Finally, perform a Laplace transform on the input image J to obtain the edge feature matrix and obtain the filtered image formula

[0024] After obtaining the image, it is necessary to further obtain the information of the captured object. Since the position of the camera is fixed, the background can be determined, and the foreground can be easily obtained through background subtraction. When there is a difference between the obtained image information and the background, the changed part is the captured object. To reduce noise and improve the effectiveness of information, when the change meets a certain threshold, it is considered that an object exists. Due to the certain differences in the scene environment, there are different thresholds at different positions. Based on this, the present invention proposes a background threshold calculation method based on scene smoothness.

[0025] Before the workplace is enabled, take the background of the environment to obtain the background image, generate its corresponding grayscale image B, and generate the zero matrix S = zeros(m, n).

[0026] For the m×n background image, set the window size to step is Translate and block the image to calculate the scene smoothness of each window.

[0027]

[0028] Since there is overlap between windows, for the smoothness of each pixel point, find the sum of the scene smoothness of all the windows to which it belongs, that is, S(x) = S(x) + smooth, x ∈ B Ω(k) , where B Ω(k)Refers to the area where the k-th window divided from the grayscale image B is located. mean() refers to the median, and avg() refers to the average value.

[0029] Finally, establish the background threshold for each pixel point

[0030] After obtaining the background threshold for each pixel point, it is necessary to extract and classify the foreground. Since in the express sorting center, the types of objects in the environment are relatively simple, only including vehicles, personnel, and express items. Due to the large volume of vehicles, they can be simply classified from the foreground area. Therefore, the present invention proposes a fast foreground classification method based on human irregularity for the difference between express items and personnel.

[0031] First, obtain the foreground image through background subtraction, and find the circumscribed matrix of the foreground object.

[0032] Then, perform binary processing on the image covered by the circumscribed matrix, where the background is 0 and the foreground is 1, to obtain a binary image P of m×n.

[0033] For the image P, there is a set of concentric pixel circles For this set of concentric pixel circles, calculate its foreground distribution as follows:

[0034] According to the foreground distribution D k , judge the foreground type, and the specific discrimination formula is as follows:

[0035]

[0036] So far, we have completed the processing of the image and the capture of object information. Next, we will obtain its motion information based on the captured objects.

[0037] When obtaining motion information, since the human body may have shaking, body rotation, and other in-place behaviors that drive the torso in a local range, if only displacement is used as the standard to judge its motion and static state, there will be a large deviation. Therefore, the present invention gives a certain redundancy to the judgment of motion and proposes a motion and static discrimination method based on a moving index.

[0038] For a given set of frame sequences, and a certain person's foreground continuously existing in this set of frame sequences First, calculate the moving index

[0039]

[0040] Among them, when M is 1, it represents that the person is in a moving state, and when it is 0, it represents that the person is in a state of staying in place. Refers to in the given foreground F PThe abscissa of a pixel in the k-th frame image is generally taken at the middle position at the bottom of the area to which it belongs. H is the height of the camera, D(x, y) refers to the horizontal distance between the pixel point at this coordinate and the camera in the on-site environment, which is a preset value, and SP is the step constant of personnel movement, generally taking 0.5 times the step length, that is, 0.7 meters.

[0041] At the same time, due to the continuous movement of objects, how to label the foreground belonging to the same object in a set of consecutive frames is also a problem. Due to the monotonicity of the sorting center environment, the regularity of the target foreground contour, and the low speed of movement, the present invention proposes a method for identifying continuous-frame objects based on the change rates of the inscribed matrix and the circumscribed matrix.

[0042] In the consecutive frames captured by the camera, when a foreground object is detected, the system labels it and calculates the size m of its circumscribed matrix C ×n C ,m C ≥n C , calculates its inscribed matrix, and obtains the midpoint coordinates C(x, y) of its inscribed matrix.

[0043] When the screen turns to the next frame, when the midpoint coordinates C′(x, y) of the inscribed matrix of the newly labeled object and the size m of the circumscribed matrix c ×n C ,m c′ ≥n c′ are satisfied and belong to the same type of object, then it can be labeled as the same object. Where v is the change rate, and the initial value is 0.25

[0044] Since the movement of the object is not uniform, therefore, according to the displacement of the object in two frames, its change rate is updated so that it can label objects with variable-speed movement. The formula is:

[0045] In the express sorting center, the present invention divides the sorting cycle of express items into four stages: to be sorted - sorting - in transit - stacking.

[0046] For the express items that have been labeled as the same foreground, obtain their frame information F = {F0, F1, F2, …, F n} within a period of time. Among them, F t refers to the information of the foreground on the background at time t within a continuous time. On this basis, obtain the movement information of the foreground during this period

[0047] M = {M 0～1 ,M 1～2 ,…,M n-1～n},M t-1～t =(F t-1(C x ,C y ),F t (C x ,C y ))), where \(t\in\{1,2,\ldots,n\}\), and among them, \(F t (C x ,C y ) refers to the midpoint coordinates of the inscribed matrix of this foreground at time \(t\). Then, the current state of this foreground is determined to see whether it is in the current state, or transfers to the next state, or an anomaly occurs.

[0048] For the express delivery in the state of waiting to be sorted, when \(M k-1～k satisfies , it is determined that it enters the sorting state. Among them, \(F k (S)\) refers to the foreground area of this frame at time \(k\), and \(U(C x ,C y ) refers to the midpoint coordinates of the inscribed matrix of the foreground of the sorter.

[0049] For the express delivery in the sorting state, after it is determined to enter the conveyor belt according to the multi-lens image stitching, it is transferred to the dedicated lens of the conveyor belt for video detection, and it is determined that it enters the state of being conveyed.

[0050] For the express delivery in the state of being conveyed, when , it is determined that it enters the stacking state. Among them, \(F Disappear (\sigma)\) refers to the position of the foreground before disappearing in consecutive frames, and \(p k refers to the coordinates of the intersection point of the conveyor belt and the collection basket within the image captured by this lens.

[0051] Due to the large scene, a single camera cannot meet the requirements. In order to accurately obtain the situation inside the sorting center, a large number of cameras need to operate jointly. In the images captured by different cameras, how to determine a continuously moving object, the present invention proposes an identification method based on boundary membership.

[0052] The cameras are hierarchically divided and regionally specified. The detection level of the high-precision small-region cameras is higher, and the boundaries of the large-region cameras and the small-region cameras coincide. At the same time, the membership of the boundary region is divided, and its formula is: Among them, \(u(x,y)\) refers to the membership of this coordinate to the region, \(x u refers to the abscissa of this point, \(x c refers to the abscissa of the center point of this viewing region. len(long) refers to the length of this viewing region, and len(width) refers to the width of this viewing region.

[0053] For the express delivery \(F\), when \(u(Fcx , F cy ) When <1, it indicates that it enters the peripheral area, F cx Refers to the abscissa of the inscribed matrix of the express delivery prospect.

[0054] For the express delivery that enters the peripheral area, within a set of consecutive frames, it satisfies Always holds and When always holds, it is said that the express delivery is moving from the area of camera S1 to the area of camera S2. When At this time, it is considered that the express delivery enters the area belonging to S2, where Refers to the membership degree of the express delivery to camera S1 at time t + 1.

[0055] The situation of the express delivery on the conveyor belt is particularly worthy of attention. At this stage, the express delivery may be blocked due to stacking, may be stuck due to morphological problems, or may fall from a non - collection basket due to inertia. For these three abnormal situations, the present invention proposes a series of identification methods.

[0056] For a set of foregrounds moving continuously on the conveyor belt, within a set of consecutive frames, it satisfies F’ t (S)>0, F t (C x , C y ) = F t+1 (C x , C y ), and when it is not unique, it is judged that a blockage abnormality occurs.

[0057] For a certain foreground moving continuously on the conveyor belt, within a set of consecutive frames, it satisfies F t (C x , C y ) = F t+1 (C x , C y ), it is determined that a jamming abnormality occurs.

[0058] For a certain foreground moving continuously on the conveyor belt, when its foreground disappears, and At this time, it is determined that a falling abnormality occurs, where F Disappear (σ) refers to the position of the foreground before it disappears in consecutive frames, and p k Refers to the coordinates of the intersection point of the conveyor belt and the collection basket in the picture captured by this lens.

[0059] During the process of express delivery sorting, due to possible mistakes in manual operations, some express deliveries may be left behind in some places, which will cause serious delays. To solve this problem, the present invention proposes a detection method for abnormally missing items under a wide - angle lens.

[0060] For the foreground of the express delivery captured by the wide-angle lens, within a set of consecutive frames, if it satisfies F t = F t+1 , where t ∈ (0, n), it is determined to be in a stationary state.

[0061] For the stationary foreground, find its circumscribed matrix, and expand it by 1.5 times with the original circumscribed matrix as the center to obtain the contour matrix F o

[0062] When the contour matrix satisfies , it is determined to be in an abnormal omission state, where F else refers to other foregrounds captured by the wide-angle matrix

[0063] Compared with the prior art, the beneficial effects of the present invention are:

[0064] Aiming at the situation of multiple light sources indoors, the anti-interference ability and denoising ability of the present invention are better.

[0065] The video detection ability of the present invention is more targeted. Only for the environment of the express delivery sorting center, the video detection accuracy of the present invention is higher and the calculation is faster.

[0066] For scenes with large fluctuations and complex environments, after the video of the present invention is processed, the edge details are richer.

[0067] The present invention deeply analyzes various situations in the express delivery sorting center and can judge various motion conditions of the express delivery in the full scene, which does not exist in the prior art. Description of the Drawings

[0068] Figure 1 is the flow chart of the present invention.

[0069] Figure 2 is the flow chart of the illumination matrix processing in the present invention.

[0070] Figure 3 is the flow chart of the adaptive high-enhanced filtering in the present invention.

[0071] Figure 4 is the flow chart of the background threshold matrix calculation in the present invention. Detailed Embodiments

[0072] The following describes in detail the embodiments of the present invention. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention and cannot be construed as limiting the present invention.

[0073] A method for identifying abnormal behaviors in a sorting center based on video detection technology proposed in this embodiment includes two parts: scene debugging and on-site detection in the specific implementation process. The specific process is as Figure 1 shown.

[0074] During the scene debugging process, it is further divided into two parts: scene content debugging and daily parameter debugging.

[0075] During the scene content debugging process, first, the corresponding cameras need to be installed, and the marginal membership degrees of each camera are set according to the marginal overlap of the camera capture ranges. At the same time, for the dedicated cameras of the conveyor belt part, the entry points of the collection baskets within the capture range of the cameras are marked.

[0076] In the daily parameter debugging part, the cameras capture images before starting work every day.

[0077] First, the illumination matrix is obtained. For details, refer to the attached drawings of the specification Figure 2 . For an input color image of m×n, its dark channel image Dark i,j = min(J C (i,j)), 0 ≤ i < m, 0 ≤ j < n C ∈ {R, G, B}, where J C (x) refers to the value of the pixel of image J at coordinate (i,j) on the C-channel component.

[0078] Set the step size step. Empirically, take step as The window size is The entire image is divided into 225 image patches. For the divided image patches Divide p,q = max(Dark i,j ),(i,j) ∈ Ω(p,q), 0 ≤ p < 15, 0 ≤ q < 15, where Ω(p,q) is the image patch in the p-th row and q-th column of the division, and (i,j) ∈ Ω(p,q) refers to the coordinates of the pixels belonging to this image patch.

[0079] Set the new window to (3, 3), perform non-overlapping translation segmentation on the matrix Divide, and obtain a set of 5×5 image groups, each image group including 9 image patches. And calculate the eigenvalue of each image group to obtain the image group feature matrix as Among them, Divide t,s refers to the image patch included in the image group in the t-th row and s-th column, refers to the i-th image patch from left to right and top to bottom in this image group.

[0080] Make corrections to Divide according to the situation of Var. The correction method is:

[0081]

[0082] Take the maximum value of the image block features covering the pixels as the atmospheric light constant at this point, and there is A i,j = max(Divide) where (i,j) in Divide. Obtain the final illumination matrix A.

[0083] Then it is the acquisition of the parameters for adaptive high-enhancement filtering. For details, please refer to the attached drawings of the specification Figure 3 For the input image J, there is B x = 0.59×J R (x) + 0.3×J G (x) + 0.11×J B (x). Among them, J C (x) refers to the value of the pixel point x of the image J on the C channel component.

[0084] Set the window size to 3×3, non-overlapping translation window, and there is Feature k = Quantile 0.99 (B x ) - avg(B x ), x∈Ω(k), where Ω(k) is the k-th partition window of B, and Quantile p (U) refers to the p-th quantile of the set U, and avg(U) refers to the average of the set U.

[0085] When more than 75% of Feature k <Quantile 0.75 (B x ) - Quantile 0.25 (B x ) 1.5 , expand the window to 1.3 times until the maximum window size σ×σ that meets the conditions is found.

[0086] Find the enhancement coefficient

[0087] Perform Laplace transform on the input image J to obtain the edge feature matrix

[0088] Finally, it is the extraction of the background threshold matrix. For details, please refer to the attached drawings of the specification Figure 4 , perform background capture on the environment, obtain the background image, generate its corresponding grayscale image B, and generate the zero matrix S = zeros(m,n).

[0089] For the m×n background image, set the window size to step is Perform translational block extraction on the image, and there is

[0090]

[0091] S(x) = S(x) + smooth, x ∈ B Ω(k) , where B Ω(k) refers to the area where the k-th window divided by the grayscale image B is located, mean() refers to the median, and avg() refers to the average value.

[0092] Establish the background threshold for each pixel point

[0093] After obtaining the atmospheric light matrix, background threshold matrix, and filtering parameters, complete the debugging of daily parameters.

[0094] During the on-site detection process, for the acquired image, first obtain its foreground according to background subtraction, and then substitute the atmospheric light matrix into the prior dark channel formula to perform defogging processing on the image.

[0095] Then according to the formula

[0096]

[0097] Enhance the image details to finally obtain the effective foreground of the image.

[0098] After obtaining the foreground, we judge the type it belongs to according to the characteristics of the foreground.

[0099] Find the circumscribed matrix of the foreground object, and perform binary processing on the image covered by the circumscribed matrix, where the background is 0 and the foreground is 1, to obtain the binary image P of m×n.

[0100] For the image P, there exists a set of concentric pixel circles For this set of concentric pixel circles, calculate its foreground distribution as:

[0101] According to the foreground distribution D k , judge the foreground type, and its specific discrimination formula is as follows:

[0102]

[0103] For personnel, we need to judge whether they are working normally according to their movement conditions.

[0104] For a given set of frame sequences, and a certain personnel foreground continuously existing in this set of frame sequences First calculate the movement index

[0105] Among them, when M is 1, it means the person is in a moving state; when M is 0, it means the person is in a stationary state. Refers to the given foreground F P The abscissa of the pixel in the k-th frame image, generally taking the middle position at the bottom of the area. H is the height of the camera, D(x, y) refers to the horizontal distance between the coordinate pixel point and the camera in the on-site environment, which is a preset value, and StepP is the pace constant of the person's movement, generally taking 0.5 times the step length, that is, 0.7 meters.

[0106] For express parcels, the present invention divides them into four parts. The state process is: to be sorted - sorting - in transit - stacking.

[0107] For the express parcels that have been labeled with the same foreground, obtain their frame information F = {F0, F1, F2,..., F n} Among them, F t Refers to the information of the foreground on the background at time t within a continuous time. On this basis, obtain the motion information of the foreground during this period

[0108] M = {M 0～1 , M 1～2 , …, M n-1～n}}, M t-1～t = (F t-1 (C x , C y ), F t (C x , C y ))), t ∈ {1, 2, …, n}, where, F t (C x , C y ) refers to the midpoint coordinates of the inscribed matrix of the foreground at time t. Then determine the current state of the foreground, determine whether it is in the current state, or transfer to the next state, or an anomaly occurs.

[0109] For the express parcels in the to-be-sorted state, when M k-1～k satisfies When, it is determined that it enters the sorting state. Among them, F k (S) refers to the foreground area of the frame at time k, and U(C x , C y ) refers to the midpoint coordinates of the inscribed matrix of the foreground of the sorting personnel.

[0110] For the express parcels in the sorting state, after determining that they enter the conveyor belt through multi-lens image stitching, they are transferred to the dedicated lens of the conveyor belt for video detection, and it is determined that they enter the in-transit state.

[0111] The express delivery in transit is considered to enter the stacking state when it meets where F Disappear (σ) refers to the position of the foreground before it disappears in consecutive frames, and p k refers to the coordinates of the intersection point of the conveyor belt and the collection basket within the captured frame of the shot.

[0112] For the express delivery moving on the conveyor belt, a method for detecting its abnormal state is proposed, and the abnormal types are as follows:

[0113] For a group of foregrounds moving continuously on the conveyor belt, within a group of consecutive frames, when F’ t (S)>0, F t (C x , C y ) = F t+1 (C x , C y ), and it is not unique, then it is judged that a blockage abnormality occurs.

[0114] For a certain foreground moving continuously on the conveyor belt, within a group of consecutive frames, when F t (C x , C y ) = F t+1 (C x , C y ), it is determined that a jamming abnormality occurs.

[0115] For a certain foreground moving continuously on the conveyor belt, when its foreground disappears, and then it is determined that a dropping abnormality occurs, where F Disappear (σ) refers to the position of the foreground before it disappears in consecutive frames, and p k refers to the coordinates of the intersection point of the conveyor belt and the collection basket within the captured frame of the shot.

[0116] For the abnormal omission of the express delivery in the picture captured by the wide-angle lens, the present invention proposes an identification method, and the steps are as follows;

[0117] For the foreground of the express delivery captured by the wide-angle lens, within a group of consecutive frames, when F t = F t+1 , t ∈ (0, n), then it is determined that it is in a static state.

[0118] For the foreground in the static state, find its circumscribed matrix, and expand it by 1.5 times with the original circumscribed matrix as the center to obtain the contour matrix F o

[0119] When the contour matrix meets then it is determined that it is in an abnormal omission state, where Felse Other foregrounds captured by the wide-angle matrix.

[0120] To achieve these goals, the present invention also proposes methods for processing some intermediate steps.

[0121] To solve the problem of object annotation between different frames, a method for continuous object recognition based on changes between frames is proposed, and its specific steps are as follows:

[0122] When a foreground object is detected, label it and find the size m of its external matrix C × n c ,m C ≥n C , find its inscribed matrix and obtain the midpoint coordinates C(x, y) of the inscribed matrix.

[0123] When the screen turns to the next frame, when the midpoint coordinates C′(x, y) of the inscribed matrix of the newly labeled object and the size m of the external matrix c ×n C ,m c′ ≥n c′ Meet and belong to the same type of object, then it can be labeled as the same object. Where v is the change rate, and the initial value is 0.25

[0124] After completing the continuous recognition, update the value of the change rate v, and the formula is:

[0125]

[0126] Repeat this process to label the objects in consecutive frames.

[0127] In view of the characteristics of a large scene and high detection accuracy requirements for some videos, a method for continuous capture of objects in different cameras and membership determination and continuity determination is proposed, and its specific steps are as follows:

[0128] Hierarchically divide the cameras and stipulate the regions. The detection level of high-precision small-region cameras is higher, and the boundaries of large-region cameras and small-region cameras coincide. At the same time, divide the membership of the boundary region, and the formula is: Among them, u(x, y) refers to the membership of the coordinate to the region, x u refers to the abscissa of the point, x c refers to the abscissa of the center point of the viewing area. len(long) refers to the length of the viewing area, and len(width) refers to the width of the viewing area.

[0129] For the express delivery F, when u(F cx ,F cy ) < 1, it indicates that it enters the surrounding area, Fcx The abscissa of the inscribed matrix indicating the prospect of the express item.

[0130] For the express item entering the peripheral area, within a set of consecutive frames, it satisfies always holds and always holds, then it is said that the express item is moving from the area of camera S1 to the area of camera S2. When then it is considered that the express item enters the area belonging to S2.

[0131] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims.

Claims

1. A method for identifying abnormal behaviors in a sorting center based on video detection technology, characterized in that: Including the following steps: Step 1: Fix the images captured by the camera. During the debugging of the scene content, first install the corresponding camera, and set the marginal membership of each camera according to the marginal overlap of the camera capture range. At the same time, for the dedicated camera of the conveyor belt part, mark the range of the collection basket entry point within its camera capture range. For the daily parameter debugging part, before starting work every day, capture images through the camera. Step 2: Perform defogging processing according to the lighting difference in the area. Step 3: Use adaptive high-enhanced filtering to improve the details of the image. Step 4: Based on the fixity of the working scene and the difference in the working area, extract the foreground by background subtraction based on the scene smoothness. Step 5: Classify the foreground of the image to judge the express parcels and personnel. Step 6: Identify the states of the express parcels in the to-be-sorted - sorting - in-transit - stacked states. Specifically, Step 6.

1. For the express items marked as the same foreground, obtain the frame information F = {F0, F1, F2, …, F n} within a period of time. Among them, F t refers to the information of the foreground on the background at time t within continuous time. On this basis, obtain the motion information M = {M 0～1 , M 1～2 , …, M n-1～n}, M t-1～t = (F t-1 (C x , C y ), F t (C x , C y ))), t ∈ {1, 2, …, n}. Among them, F t (C x , C y ) refers to the midpoint coordinates of the inscribed matrix of the foreground at time t. Then, determine the current state of the foreground to determine whether it is in the current state, or transfer to the next state, or an anomaly occurs. Step 6.

2. For the express parcels in the to-be-sorted state, when M k-1～k satisfies , it is determined that they enter the sorting state. Among them, F k (S) refers to the foreground area of this frame at time k, and U(C x , C y ) refers to the midpoint coordinates of the foreground inscribed matrix of the sorting personnel; Step 6.3: For the express parcels in the sorting state, after determining that they enter the conveyor belt through multi-lens image stitching, switch to the dedicated lens of the conveyor belt for video detection and determine their in-transit state. At the same time, in the application of multi-lens image stitching, due to the large scene and high requirements for video detection accuracy, it is necessary to continuously capture an object in different cameras and perform membership determination and continuous determination. The specific steps are as follows: Step 6.3.1: Hierarchically divide the cameras and define regions. The detection level of the high-precision small-region cameras is higher, and the boundaries of the large-region cameras and the small-region cameras coincide. At the same time, perform membership degree division on the boundary regions, and its formula is: where u(x,y) refers to the membership degree of the coordinate to the region, and x u refers to the abscissa of the point, x c refers to the abscissa of the center point of the viewing area, len(long) refers to the length of the viewing area, and len(width) refers to the width of the viewing area; Step 6.3.

2. For express delivery F, when u(F cx , F cy ) < 1, it indicates that it enters the peripheral area. F cx refers to the abscissa of the inscribed matrix of the foreground of the express delivery; Step 6.3.

3. For the express items entering the peripheral area, when they satisfy always holds and always holds, it is said that the express item is moving from the area of camera S1 to the area of camera S2. When is satisfied, it is considered that the express item enters the area of S2; Step 6.

4. For the express delivery in transit, when it satisfies , it is determined that it enters the stacking state, where F Disappear (σ) refers to the position before the foreground disappears in consecutive frames, and p k refers to the coordinates of the intersection point of the conveyor belt and the collection basket within the frame captured by the camera; and, it is necessary to identify the abnormal situations that occur for the express delivery moving on the conveyor belt. At the same time, during the process of sorting express deliveries, due to manual operation errors, express deliveries may be missed. It is necessary to detect the abnormally missed items under the wide-angle lens. The specific steps are as follows: Step 6.4.

1. For a group of foregrounds moving continuously on the conveyor belt, within a group of consecutive frames, if F’ t (S)>0, F t (C x , C y ) = F t+1 (C x , C y ), and it is not unique, then it is determined that a blockage exception occurs; Step 6.4.

2. For a certain foreground moving continuously on the conveyor belt, within a set of consecutive frames, if F t (C x , C y ) = F t+1 (C x , C y ), it is determined that a jamming abnormality has occurred; Step 6.4.

3. For a certain foreground moving continuously on the conveyor belt, when its foreground disappears and at this time, it is determined that a dropping anomaly occurs, where F Disappear (σ) refers to the position of the foreground before it disappears in consecutive frames, and p k refers to the coordinates of the intersection point of the conveyor belt and the collection basket within the frame captured by this camera; Step 6.4.

4. For the foreground of the express delivery captured by the wide-angle lens, within a set of consecutive frames, if it satisfies F t = F t+1 , where t ∈ (0, n), then it is determined to be in a stationary state; Step 6.4.5: For the foreground in the static state, find its circumscribed matrix and expand it by 1.5 times with the original circumscribed matrix as the center to obtain the contour matrix F O ; Step 6.4.

6. When the contour matrix satisfies , it is determined to be in an abnormal omission state, where F else refers to other foregrounds captured by the wide-angle matrix; Step 7: Identify the anomalies of the express parcels in different states.

2. The method for identifying abnormal behaviors in a sorting center based on video detection technology according to claim 1, characterized in that: In Step 2, the defogging processing is calculated using the local lighting conditions for the point light source situation in the sorting center. The specific steps are as follows: Step 2.1: For an input color image of m×n, calculate its dark channel map Dark i,j = min(J C (j,j)), 0 ≤ i < m, 0 ≤ j < n, C ∈ {R, G, B}, where J C (x) represents the value of the pixel at coordinate (i, j) in image J on the C-channel component; Step 2.2: Set the step size step. Empirically, step is taken as The window size is Divide the entire image into 225 image patches, and there is the divided image patch Divide p,q = max(Dark i,j ),(i, j) ∈ Ω(p, q), 0 ≤ p < 15, 0 ≤ q < 15; where Ω(p, q) is the image patch in the p-th row and q-th column of the division, and (i, j) ∈ Ω(p, q) means the coordinates of the pixel belonging to this image patch; Step 2.3: Set the new window as (3, 3), perform non-overlapping translation segmentation on the matrix Divide to obtain a set of 5×5 image groups, each image group including 9 image blocks; and calculate the eigenvalues of each image group to obtain the image group feature matrix as where, Divide t,s refers to the image blocks included in the image group in the t-th row and s-th column, refers to the i-th image block from left to right and top to bottom in this image group; Step 2.4: Make a correction to Divide according to the situation of Var. The correction method is: Step 2.5: Take the maximum value of the image block features covering the pixels as the atmospheric light constant at this point, and there is A i,j = max(Divide) where (i, j) in Divide, to obtain the final illumination matrix A.

3. The method for identifying abnormal behaviors in a sorting center based on video detection technology according to claim 1, characterized in that: The specific method of Step 3 is: Step 3.1: For the input image J, there is B x = 0.59 × J R (x) + 0.3 × J G (x) + 0.11 × J B (x); where J C (x) refers to the value of the pixel point x of the image J on the C channel component; Step 3.2: Set the window size to 3×3, a non-overlapping translation window, and there is Feature k = Quantile 0.99 (B x ) - avg(B x ), x ∈ Ω(k), where Ω(k) is the k-th partition window of B, and Quantile p (U) refers to the p-th quantile of the set U, and avg(U) refers to the average of the set U; Step 3.3: When more than 75% of the Feature k <Quantile 0.75 (B x )-Quantile 0.25 (B x ) 1.5 , expand the window to 1.3 times until the maximum window size σ×σ that meets the conditions is found; Step 3.4, obtain the enhancement coefficient Step 3.5: Perform Laplace transform on the input image J to obtain an edge feature matrix C ∈ {R, G, B}; Step 3.6, calculate the filtered image C ∈ {R, G, B} 4. The recognition method of abnormal behaviors in a sorting center based on video detection technology according to claim 1, characterized in that: Step 4.1: Before the workplace is enabled, take the background of the environment to obtain the background image, generate its corresponding grayscale image B, and generate a zero matrix S = zeros(m,n). Step 4.2: For the m×n background image, set the window size to The step is Perform translation and block extraction on the image, and there is S(x) = S(x) + smooth, x ∈ B Ω(k) , where B Ω(k) refers to the region where the k-th window divided from the grayscale image B is located, mean() refers to the median, and avg() refers to the average value; Step 4.3, establish the background threshold for each pixel point 5. The recognition method of abnormal behaviors in a sorting center based on video detection technology according to claim 1, characterized in that: After obtaining the background threshold of each pixel point, it is necessary to extract and classify the foreground. According to the monotonicity of the foreground in the sorting center, there are only three types of objects: vehicles, personnel, and express parcels. And because the volume of the vehicle is much larger than the other two types, a simple classification can be made from the foreground area. Therefore, a method for distinguishing the foreground areas of personnel and express parcels in the sorting center scene based on the irregularity of the human body contour is proposed. The specific steps are as follows: Step 5.1: Obtain the foreground image through background subtraction and find the circumscribed matrix of the foreground object. Step 5.2: Perform binary processing on the image covered by the circumscribed matrix, where the background is 0 and the foreground is 1, to obtain a binary image P of m×n. Step 5.

3. For image P, there exists a set of concentric pixel circles For this set of concentric pixel circles, calculate its foreground distribution as follows: Step 5.

4. According to the foreground distribution D k , determine the foreground type, and the specific discrimination formula is as follows:

6. The recognition method of abnormal behaviors in a sorting center based on video detection technology according to claim 5, characterized in that: In Step 5, it is necessary to judge whether the individuals classified as personnel are in a moving or stationary state. The specific method is as follows: For a given set of frame sequences and a certain foreground of a person continuously present in the set of frame sequences First, calculate the movement index Among them, when M is 1, it means the person is in a moving state, and when it is 0, it means the person is in a stationary state; refers to the given foreground F P The abscissa of the pixel in the k-th frame image, generally taking the middle position at the bottom of the area; H is the height of the camera, D(x, y) refers to the horizontal distance between the coordinate pixel point and the camera in the on-site environment, which is a preset value, and StepP is the step constant of the person's movement, generally taking 0.5 times the step length, that is, 0.7 meters; Due to the continuous movement of the object, it is necessary to label the foreground belonging to the same object in a group of consecutive frames. The specific content is: In consecutive frames captured by a camera, when a foreground object is detected, it is labeled, and the size m of its external matrix is obtained C ×n C ,m C ≥n C , the inscribed matrix is obtained, and the midpoint coordinates C(x, y) of the inscribed matrix are obtained; when the screen turns to the next frame, when the midpoint coordinates C′(x, y) of the inscribed matrix of the newly labeled object and the size m of the external matrix C ×n C ,m C′ ≥n C′ meet and belong to the same class of objects, then they can be labeled as the same object; where v is the change rate, and the initial value is 0.25; after continuous recognition is completed, the value of the change rate v is updated, and the formula is:

Images