Storage workshop safety monitoring method and system based on image recognition

Abstract

The invention relates to the technical field of image processing, in particular to a storage workshop safety monitoring method and system based on image recognition. The method comprises the following steps: acquiring a first rectangle corresponding to a stone stack and a second rectangle corresponding to dust smoke in each frame of storage workshop image by using a neural network; according to the distribution of the edge pixel points in the first rectangle, stone stacking characteristics of the first rectangle are determined; determining the similarity of stone stacks between different frames according to the similarity between the stone stack characteristics; determining a first possibility of collapse of the stone stacks according to the similarity of the stone stacks among the continuous multiple frames and the change of the first rectangles; determining a second possibility of collapse of the stone stack according to the second rectangle and the first possibility; and giving a safety alarm when the second possibility is greater than a collapse threshold value. The stone stacking storage condition in the storage workshop is monitored accurately, and the life safety of field workers in the storage workshop is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a warehouse safety monitoring method and system based on image recognition. Background Art

[0002] Stone storage workshops are facilities dedicated to storing and managing various stone materials (such as crushed stone, sandstone, gravel, etc.). Stones in stone storage workshops are usually stored in piles, and roads are left between the piles for inspection by patrol personnel. In large stone storage workshops, the stone piles are usually very huge and very easy to collapse. However, due to limited human vision, it is impossible to detect the collapse of stone piles in the initial stage in time, and then escape from the storage workshop, endangering the lives of relevant staff.

[0003] At present, neural networks are usually used to identify stone piles in warehouse images taken from a bird's-eye view, and to determine whether the stone piles have collapsed based on the changes in the area of ​​the stone piles in the image. For example, the paper (Redmon, Joseph, et al. "You Only Look Once: Unified, Real-Time Object Detection." Computer Vision & Pattern Recognition IEEE, 2016.) discloses a neural network model that can be used to accurately identify stone piles in warehouse images.

[0004] In the initial stage of the collapse of the stone pile, only the top of the stone pile collapses, and the area of ​​the stone pile in the warehouse workshop image taken from a bird's-eye view will not change. If the collapse of the stone pile is judged only based on the change in the area of ​​the stone pile in the warehouse workshop image, the collapse warning may be delayed. When the stone pile collapses in a large area, it may be too late to issue a warning, which may cause the on-site staff of the stone warehouse workshop to be unable to escape in time, posing a hidden danger to the safety of the staff of the stone warehouse workshop. Summary of the invention

[0005] The present invention provides a storage workshop safety monitoring method and system based on image recognition, aiming to solve the technical problem of hysteresis in identifying collapse according to the change of stone stacking area in an image in the related art.

[0006] In a first aspect, the present invention provides a warehouse safety monitoring method based on image recognition. The method comprises: A neural network is used to obtain the first rectangle corresponding to the stone pile and the second rectangle corresponding to the dust and smoke in each frame of the warehouse image; the stone pile feature of the first rectangle is determined according to the distribution of edge pixels in the first rectangle; the similarity of the stone piles between different frames is determined according to the similarity between the stone pile features corresponding to the first rectangle in different frames of warehouse images; the first possibility of the collapse of the stone pile at the current moment is determined according to the similarity of the stone piles between the warehouse image at the current moment and the warehouse images of several previous frames and the change of the first rectangle, and the first possibility is negatively correlated with the similarity of the stone pile and negatively correlated with the current moment. The area ratio of all first rectangles in the warehouse image and the difference between the areas of all first rectangles in the warehouse image at the current moment and several frames of warehouse images before are positively correlated; the second possibility of the collapse of the stone pile is determined based on the second rectangles in the warehouse image at the current moment and the first possibility, and the second possibility is positively correlated with the first possibility and the area ratio of all second rectangles in the warehouse image at the current moment, and is positively correlated with the difference between the area of ​​the minimum circumscribed rectangle of all second rectangles and the area of ​​all second rectangles; in response to the second possibility of the collapse of the stone pile being greater than the collapse threshold, a safety alarm is issued.

[0007] The effect is as follows: the present invention combines the similarity between the stone pile features of the first rectangle in each frame with the size and shape changes of the first rectangle to obtain the first possibility of the stone pile collapse, and can promptly discover the collapse in the early stage of the stone pile collapse and issue a safety alarm, which is beneficial to ensuring the safety of the staff of the stone storage workshop; the present invention combines the changes in the stone pile and dust and smoke in the storage workshop to predict the possibility of large-scale coordinated collapse of the stone pile, and can measure the possibility of stack collapse from various stages of the development of the stack collapse. When a large-scale stone pile collapse is about to occur, a pre-alarm can be issued in time, further ensuring the life safety of on-site staff of the stone storage workshop.

[0008] The determining of the stone stacking features of the first rectangle includes: adjusting the image within the first rectangle to The fixed size of the first rectangle is used to detect the edge of the adjusted first rectangle to obtain an edge image; the stone stacking features of the first rectangle are determined according to the edge image. : , is the preset side length; , , They represent the edge images Row, No. Row, No. The cumulative sum of the grayscale values ​​of the pixels in the row; , , They represent the edge images Column, No. Column, No. The cumulative sum of the grayscale values ​​of the pixels in the column; , , They represent the edge images Row, No. Row, No. The gray value of the row is The mean of the horizontal coordinates of the pixels; , , They represent the edge images Column, No. Column, No. The gray value of the column is The mean of the vertical coordinates of the pixels.

[0009] The effect is that the positions of stones in the stone pile are random, and when the stones in the first rectangle are in different positions, the edges generated will also present different shapes. Therefore, the present invention reflects the characteristics of the stone pile according to the cumulative sum of the grayscale values ​​of each row of pixels in the edge image, the cumulative sum of the grayscale values ​​of each column of pixels, the mean of the horizontal coordinates of each row of edge pixels, and the mean of the vertical coordinates of each column of edge pixels in the edge image, so that the stone pile characteristics corresponding to the first rectangle in different frames can reflect the changes of the stone pile, which is conducive to timely discovering the collapse of the stone pile in the early stage of the collapse.

[0010] Preferably, the determining of the similarity of the stone stacks between different frames includes: taking the storage workshop images of several frames before the storage workshop image at the current moment as a reference image respectively; and determining the similarity of the stone stacks between the storage workshop image at the current moment and each reference image: , The image of the warehouse at the current moment and the Similarity of stone piles between reference images; Indicates the current moment in the warehouse image The first rectangular stone pile features, Take all Integers in Indicates the number of first rectangles in the warehouse workshop image at the current moment; The current warehouse image The first rectangle is in A first rectangular stone stack feature corresponding to a reference image; represents the cosine similarity function; Represents the minimum function.

[0011] Preferably, the first possibility of the stone pile collapse at the current moment satisfies the expression: ;in, Indicates the first possibility of the stone pile collapsing at the current moment; Represents the sum of the areas of all first rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; The image of the warehouse at the current moment and the The similarity of stone stacks between reference images, Take all Integers in represents a preset first quantity; Indicates The sum of the areas of all first rectangles in the reference images; represents the natural exponential function.

[0012] The effect is that the present invention combines the changes in the size and shape of the first rectangle corresponding to the stone pile in different frames and the similarity between the stone pile features of the first rectangle to obtain the first possibility of the stone pile collapse, thereby avoiding the situation that the size and shape of the first matrix may not change in the early stage of the stone pile collapse, resulting in the stone pile collapse cannot be discovered in time. The present invention can detect the collapse in time at the early stage of the stone pile collapse and issue a safety alarm, which is beneficial to ensuring the safety of the staff of the stone storage workshop.

[0013] Preferably, the second possibility of the stone pile collapse satisfies the expression: ;in, Indicates the second possibility of the collapse of the stone pile; Represents the sum of the areas of all second rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; Indicates the area of ​​the minimum circumscribed rectangle of all second rectangles in the warehouse workshop image at the current moment; Indicates the first possibility of the stone pile collapsing at the current moment.

[0014] The effect is that the present invention utilizes the size and shape changes of the second rectangle corresponding to the dust and smoke to reflect the diffusion of the dust and smoke, and combines the diffusion of the dust and smoke and the first possibility of the stone pile collapse to predict whether the stone pile collapse will cause a linked collapse, and the result is more accurate.

[0015] Preferably, the second possibility of the stone pile collapse satisfies the expression: ;in, It indicates the second possibility of the collapse of the stone pile, reflecting the possibility of the linkage collapse of the stone pile; Represents the sum of the areas of all second rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; Indicates the area of ​​the minimum circumscribed rectangle of all second rectangles in the warehouse workshop image at the current moment; Indicates the wind speed at the current moment; represents a hyperparameter; Indicates the first possibility of the stone pile collapsing at the current moment; represents the natural exponential function.

[0016] The effect is that in the process of calculating the second possibility of the collapse of the stone pile, the present invention eliminates the influence of the wind speed in the storage workshop on the diffusion of dust and smoke, so that the obtained result can better reflect the possibility of the linked collapse of the stone pile.

[0017] Preferably, the method further includes: in response to the absence of a second rectangle in the storage workshop image at the current moment, if the first possibility of the stone pile collapse at the current moment is greater than the collapse threshold, issuing a safety alarm.

[0018] Preferably, the neural network adopts the YOLOv5 model.

[0019] In a second aspect, the present invention proposes a warehouse workshop safety monitoring system based on image recognition, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement a warehouse workshop safety monitoring method based on image recognition as described in any one of the above invention contents.

[0020] The present invention has the following beneficial effects: the present invention can timely discover the collapse of the stone pile in the early stage by obtaining the first possibility of the collapse of the stone pile. The present invention predicts the possibility of large-scale linked collapse of the stone pile in combination with the changes in dust and smoke. The possibility of the stack collapse can be measured from each stage of the development of the stack collapse and an alarm can be issued in time, which is beneficial to protecting the lives of on-site workers in the stone storage workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 It is a flowchart of the steps of a warehouse workshop safety monitoring method based on image recognition according to an embodiment of the present invention; Figure 2 is a schematic diagram of the minimum circumscribed rectangle of all second rectangles; Figure 3 It is a structural block diagram of a warehouse safety monitoring system based on image recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 It is a flowchart schematically showing the steps of a warehouse workshop safety monitoring method based on image recognition according to an embodiment of the present invention.

[0025] like Figure 1 As shown, a warehouse workshop safety monitoring method based on image recognition includes steps S1 to S5.

[0026] Step S1: Collect multiple frames of warehouse images, and use a neural network to obtain a first rectangle corresponding to the stone pile and a second rectangle corresponding to the dust and smoke in each frame of the warehouse image.

[0027] Stones are stored in the storage workshop. The images of the storage workshop are collected in real time through a camera with a high altitude bird's-eye view inside the storage workshop, and a neural network is used to identify the stone piles and dust and smoke in the images of the storage workshop.

[0028] In one embodiment, the specific contents of the neural network include: the neural network adopts the YOLOv5 network; the input of the neural network is the warehouse image, and the output is the bounding box corresponding to the stone pile and the bounding box corresponding to the dust and smoke in the warehouse image; the data set used by the neural network is a data set composed of images containing stone piles or dust and smoke at various high-altitude bird's-eye view angles, and the labels are the bounding box of the stone pile and the bounding box of the dust and smoke in the image containing the stone pile or dust and smoke; the loss function of the neural network is the cross entropy loss function and the intersection over union (IOU) loss function.

[0029] Among them, YOLOv5 is a neural network algorithm of the You Only Look Once (YOLO) target detection model series. YOLOv5 integrates the architecture design based on YOLO. The cross entropy loss function and the IOU loss function are commonly used loss functions in YOLOv5. Implementers can also choose a neural network model based on actual implementation conditions.

[0030] Input each frame of the warehouse image into the trained neural network to obtain the bounding box corresponding to the stone pile and the bounding box corresponding to the dust and smoke in the warehouse image. The bounding box corresponding to the stone pile is recorded as the first rectangle, and the bounding box corresponding to the dust and smoke is recorded as the second rectangle. Since there may be multiple stone piles in the warehouse, there may be multiple first rectangles and multiple second rectangles in the warehouse image.

[0031] So far, the first rectangle corresponding to the stone pile and the second rectangle corresponding to the dust and smoke in each frame of the storage workshop image are obtained.

[0032] Step S2: determining the stone stacking features of the first rectangle according to the distribution of edge pixel points in the first rectangle.

[0033] It should be noted that improper stacking of stone piles in a storage workshop may lead to collapse. In the early stage of the collapse of the stone pile, a local collapse will occur on the top of the stone pile, and the local collapse will gradually spread from the top of the pile until the entire pile collapses. During the collapse of the stone pile, the positions of the stones in the stone pile change. When the stones in the first rectangle are in different positions due to collapse, the edges generated will also present different shapes. Therefore, the present invention determines the stone pile features of the storage workshop image according to the distribution of edge pixel points in the first rectangle in the storage workshop image.

[0034] Specifically, each first rectangle in the warehouse image corresponds to a stone pile. For any first rectangle in any frame of the warehouse image, in order to facilitate subsequent calculations, the image in the first rectangle is adjusted to The fixed size is recorded as the stone stack image. It is the preset side length, which is set by the implementer according to the actual implementation situation and is not limited to any specific length. For example, When the size of the first rectangle exceeds When the size of the first rectangle is smaller than , the image in the first rectangle is adjusted by downsampling.

[0035] In one embodiment, the Canny operator is used to perform edge detection on the stone pile image to obtain the edge image of the stone pile image, wherein the edge image is a binary image, and the pixel points with a gray value of 1 in the edge image are edge pixel points in the stone pile image, and the pixel points with a gray value of 0 are non-edge pixel points in the stone pile image. In other embodiments, the implementer can select an edge detection algorithm according to the actual implementation situation, such as the Sobel operator.

[0036] In one embodiment, the stone pile feature of the first rectangle is determined according to the grayscale distribution of the pixel points in the edge image: ; In the formula, The stone pile feature representing the first rectangle; is the preset side length; , , They represent the edge images Row, No. Row, No. The cumulative sum of the grayscale values ​​of the pixels in the row; , , They represent the edge images Column, No. Column, No. The cumulative sum of the grayscale values ​​of the pixels in the column; , , They represent the edge images Row, No. Row, No. The gray value of the row is The mean of the horizontal coordinates of the pixels; , , They represent the edge images Column, No. Column, No. The gray value of the column is The mean of the vertical coordinates of the pixels.

[0037] Since the positions of the stones in the stone pile in the first rectangle are random, the edges generated when the stones in the first rectangle are in different positions will also present different shapes. Therefore, there is a certain difference between the cumulative sum of the grayscale values ​​of each row of pixels in the edge image corresponding to the stone pile in the first rectangle before collapse and the cumulative sum of the grayscale values ​​of each row of pixels in the edge image corresponding to the collapse process. It reflects the cumulative sum of the grayscale values ​​of each row of pixels in the edge image corresponding to the first rectangle. Similarly, there is a certain difference between the cumulative sum of the grayscale values ​​of each column of pixels in the edge image corresponding to the collapse and the edge image corresponding to the collapse process. It reflects the cumulative sum of the grayscale values ​​of each column of pixels in the edge image corresponding to the first rectangle. Similarly, when the stones in the first rectangle are in different positions, the edges they produce will also present different shapes, and the coordinates of the corresponding edge pixels will also be in different distributions. and It can also represent the uniqueness of the stone in the first rectangle. , , as well as Combine them to get the stone stacking features of the first rectangle .

[0038] In another embodiment, the stone stacking feature of the first rectangle is determined according to the grayscale distribution of the pixels in the edge image: ; In the formula, The stone pile feature representing the first rectangle; is the preset side length; , , They represent the edge images Row, No. Row, No. The cumulative sum of the grayscale values ​​of the pixels in the row; , , They represent the edge images Column, No. Column, No. The cumulative sum of the grayscale values ​​of the pixels in the column; , , They represent the edge images Row, No. Row, No. The gray value of the row is The mean of the horizontal coordinates of the pixels; , , They represent the edge images Column, No. Column, No. The gray value of the column is In order to further improve the characteristic difference of stone piles in different states, this embodiment will , , as well as Combine to get the characteristics of stone stacking In this embodiment, the characteristics of the stone pile are calculated by taking the index and then subtracting it. The implementation personnel can adjust it according to the actual situation. , , as well as combination method.

[0039] Similarly, the stone stacking features of each first rectangle in each frame of the storage workshop image are obtained.

[0040] Step S3: Determine the similarity of the stone piles between different frames based on the similarity between the stone pile features corresponding to the first rectangle in the warehouse images of different frames; determine the first possibility of the collapse of the stone pile at the current moment based on the similarity between the warehouse image at the current moment and the warehouse images of several previous frames and the change of the first rectangle.

[0041] It should be noted that when the stone pile collapses as a whole, the size and shape of the first rectangle corresponding to the stone pile in the storage workshop images of the previous and next frames will change significantly. In order to accurately detect whether the stone pile collapses, the present invention obtains the first possibility of the stone pile collapse through the changes in the size and position of the first rectangle in the storage workshop image.

[0042] Specifically, the center point of each first rectangle in each frame of the warehouse workshop image is obtained. For the target first rectangle in the warehouse workshop image at the current moment, the first rectangle whose center point in any frame of the warehouse workshop image is the smallest distance from the center point of the target first rectangle is used as the first rectangle corresponding to the target first rectangle in the warehouse workshop image. Similarly, the first rectangle corresponding to each first rectangle in the warehouse workshop image at the current moment in each frame of the warehouse workshop image is obtained.

[0043] The warehouse image before the current moment The frame warehouse workshop image is used as a reference image respectively, where Indicates the preset first quantity. The implementer can set the first quantity according to the actual implementation situation. For example According to the similarity between the stone stacking features of each first rectangle in the warehouse image at the current moment and the corresponding first rectangle in the reference image, the similarity of the stone stacking between the warehouse image at the current moment and the reference image is determined: ; in, The image of the warehouse at the current moment and the Similarity of stone piles between reference images; Indicates the current moment in the warehouse image The first rectangular stone pile features, Take all Integers in Indicates the number of first rectangles in the warehouse workshop image at the current moment; The current warehouse image The first rectangle is in A first rectangular stone stack feature corresponding to a reference image; represents the cosine similarity function; represents the minimum function; since the value range of cosine similarity is [-1,1], the present invention uses Map the cosine similarity to the range of [0,1] for subsequent calculations. The greater the cosine similarity between the stone stack features of the corresponding first rectangles in the two frames of the warehouse image, the smaller the possibility of the stone stack collapse. On the contrary, the smaller the cosine similarity between the stone stack features of the corresponding first rectangles in the two frames of the warehouse image, the greater the possibility of the stone stack collapse. The present invention takes the minimum value of the cosine similarities between all the corresponding first rectangular stone stack features in the two frames of the warehouse image as the similarity of the stone stack between the two frames of the warehouse image.

[0044] According to the current warehouse workshop image and the previous continuous The change of the first rectangle in the frame storage workshop image determines the first possibility of the stone pile collapse at the current moment: ; in, Indicates the first possibility of the stone pile collapsing at the current moment; Represents the sum of the areas of all first rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; The image of the warehouse at the current moment and the The similarity of stone stacks between reference images, Take all Integers in represents a preset first quantity; Indicates The sum of the areas of all first rectangles in the reference images; represents the natural exponential function; Represents the minimum function.

[0045] The image of the storage workshop is taken from a high altitude, and there are usually multiple stone piles in the storage workshop. When the stone piles have not collapsed, the area of ​​the first rectangle corresponding to each stone pile is small. When the stone piles collapse, the area of ​​the corresponding rectangles corresponding to each stone pile gradually increases, and multiple scattered stone piles will gradually merge into one stone pile. Indicates the area ratio of all the first rectangles in the warehouse image at the current moment, reflecting the size of the stone stack in the warehouse image at the current moment. When the value is larger, the stone pile is more likely to collapse at the current moment; on the contrary, when The smaller it is, the less likely the stone pile will collapse at the current moment. Only the size of the stone pile in the warehouse image at the current moment is considered, and the motion state of the stone pile cannot be measured. Therefore, the present invention measures the motion state of the stone pile by the size change of the first rectangle corresponding to the stone pile in a period of time before the current moment. Make corrections, It reflects the size change of the first rectangle corresponding to the stone pile. The larger the value is, the faster the stone pile changes in the period before the current moment. The stone pile is more likely to collapse at the current moment. right The greater the expansion, the greater the possibility of the stone pile collapsing; conversely, The smaller the value is, the more stable the stone pile is in the period before the current moment, and the lower the possibility of the stone pile collapsing at the current moment. right The smaller the expansion, the more the first probability of the stone pile collapsing depends on .

[0046] It reflects the minimum similarity of the stone pile between the current warehouse image and the warehouse image in the previous period of time. In the early stage of the collapse of the stone pile, the top of the stone pile will partially collapse. Since the warehouse image is taken from a high altitude, when the top of the stone pile partially collapses, the first rectangle corresponding to the stone pile may not change. Therefore, in order to identify the partial collapse of the stone pile, the present invention uses the minimum similarity of the stone pile between the current warehouse image and the warehouse image in the previous period of time. To further When the minimum similarity is smaller, it means that the characteristics of the stone pile at the current moment have changed significantly compared with the previous stone pile. This change is more likely to be caused by the collapse of the stone pile. right The greater the expansion, the greater the first possibility of the stone pile collapse; conversely, when the minimum similarity is larger, it means that the stone pile characteristics at the current moment are basically unchanged compared with the previous stone pile, and the possibility of the stone pile collapse is smaller. right The smaller the expansion.

[0047] At this point, the first possibility of the stone pile collapse at the current moment is obtained.

[0048] Step S4: determining the second possibility of the stone pile collapse according to the second rectangle in the storage workshop image at the current moment and the first possibility of the stone pile collapse at the current moment.

[0049] It should be noted that, according to the change of the first rectangle corresponding to the stone stack, the first possibility of the collapse of the stone stack can be reflected. However, when the stone stack in the storage workshop collapses, they usually do not all collapse at the same time. Instead, one stack collapses first and then drives the other stacks to collapse in conjunction. Since the stone stack often contains a large amount of dust, when the stack collapses, a certain amount of dust and smoke will be generated. The diffusion of the dust and smoke occurs earlier than the coordinated collapse of the stacks. Therefore, in order to monitor whether the stone stack is about to collapse in conjunction before the coordinated collapse, the present invention calculates the second possibility of the collapse of the stone stack at the current moment according to the second rectangle corresponding to the dust and smoke in the storage workshop image and the first possibility of the stone stack collapse at the current moment, and uses the second possibility to reflect the possibility of the coordinated collapse of the stone stack.

[0050] In one embodiment, in response to the presence of a second rectangle in the warehouse image at the current moment, the second possibility of the collapse of the stone pile is calculated according to the second rectangle corresponding to the dust and smoke in the warehouse image at the current moment and the first possibility of the collapse of the stone pile at the current moment: ; in, It indicates the second possibility of the collapse of the stone pile, reflecting the possibility of the linkage collapse of the stone pile; Represents the sum of the areas of all second rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; Represents the area of ​​the minimum circumscribed rectangle of all second rectangles in the warehouse image at the current moment, Figure 2 is a schematic diagram of the minimum circumscribed rectangle of all second rectangles, Figure 2 The medium grey part is the second rectangle; Indicates the first possibility of the stone pile collapsing at the current moment.

[0051] Indicates the proportion of the second rectangle corresponding to the dust and smoke in the warehouse image at the current moment, The larger the value is, the more dust and smoke there is at the current moment. The dust and smoke are more likely to be caused by the partial collapse of the stone pile, and the subsequent linkage collapse of the stone pile is more likely to occur. The smaller it is, the less dust and smoke there is at the current moment, and the smaller the possibility of linked collapse of the stone pile. It reflects the diffusion degree of dust and smoke at the current moment. The larger the value, the wider the diffusion range of dust and smoke, and the more likely the dust and smoke is caused by the collapse of stone piles; conversely, The smaller the value, the smaller the diffusion range of dust and smoke, and the smaller the possibility of the stone pile linkage collapse. Since the first rectangle corresponding to the stone pile has a hysteresis in the detection of linkage collapse, when the first possibility of stone pile collapse obtained by the first rectangle corresponding to the stone pile is small, a large amount of dust and smoke may have actually been generated to form a large-scale pile collapse. Therefore, the present invention uses right Make an upward correction so that The larger the size, the second probability of the stone pile collapsing. The bigger.

[0052] It should be further explained that, in order to maintain ventilation in the storage workshop, fans are usually installed in the storage workshop, so that the wind speed in the storage workshop has a certain impact on the dust and smoke. When the wind speed is high, the dust and smoke that originally had a small range may be blown to a larger range. Therefore, in another embodiment of the present invention, the second possibility of stone pile collapse is calculated in combination with the wind speed.

[0053] In another embodiment, a wind speed sensor is deployed in the storage workshop, and the wind speed sensor is used to collect the wind speed at each moment in real time. In response to the presence of a second rectangle in the storage workshop image at the current moment, the second possibility of the stone pile collapse is calculated according to the second rectangle corresponding to the dust and smoke in the storage workshop image at the current moment, the wind speed at the current moment, and the first possibility of the stone pile collapse at the current moment: ; in, It indicates the second possibility of the collapse of the stone pile, reflecting the possibility of the linkage collapse of the stone pile; Represents the sum of the areas of all second rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; Indicates the area of ​​the minimum circumscribed rectangle of all second rectangles in the warehouse workshop image at the current moment; Indicates the wind speed at the current moment; Represents a hyperparameter used to obtain wind speed The relative size of the velocity is 4 m / s according to experience, and the implementer can set it according to the actual implementation situation; Indicates the first possibility of the stone pile collapsing at the current moment; represents the natural exponential function.

[0054] Indicates the proportion of the second rectangle corresponding to dust and smoke in the warehouse workshop image at the current moment. It reflects the diffusion degree of dust and smoke at the current moment. It represents the wind speed influence factor. Considering the influence of wind speed in the storage workshop, when the wind speed is high, the dust and smoke that were originally small in range may be blown to a larger range, which makes the possibility of linkage collapse of the stone pile judged based on the diffusion degree of dust and smoke have a certain deviation. Therefore, the wind speed influence factor is used to The degree of diffusion Make corrections when The larger the Adjustment makes Reduce, thereby reducing the impact of wind speed on the possibility of rock pile collapse; The smaller the size, the more likely the dust and smoke diffusion is caused by the collapse of the stack. The smaller the time, the The smaller the degree of correction, the The closer the result is Since the first rectangle corresponding to the stone pile has a hysteresis for the detection of the linkage collapse, when the first possibility of the stone pile collapse obtained by the first rectangle corresponding to the stone pile is small, a large amount of dust and smoke may have been generated to form a large-scale pile collapse. right Make an upward correction so that The larger the size, the second probability of the stone pile collapsing. The bigger.

[0055] At this point, the second possibility of stone pile collapse was obtained.

[0056] Step S5: In response to the second possibility of the stone pile collapse being greater than the collapse threshold, a safety alarm is issued.

[0057] Among them, the safety alarm includes: sending out sound and light alarm signals, and sending out distress signals.

[0058] In one embodiment, in response to the presence of a second rectangle in the storage workshop image at the current moment, when the second possibility of the stone pile collapse is greater than a preset collapse threshold, a buzzer sends a sound alarm signal to remind on-site staff to evacuate, and a red LED quickly flashes to send a light alarm signal. A distress signal can also be sent to the fire department through a wireless network.

[0059] In response to the absence of the second rectangle in the storage workshop image at the current moment, when the first possibility of the stone pile collapse is greater than a preset collapse threshold, an audible and visual alarm signal and a distress signal are issued.

[0060] The collapse threshold is set by the implementers based on the actual implementation situation and is not limited to a specific value, for example, 0.6.

[0061] Figure 3 The figure schematically shows a structural block diagram of a warehouse safety monitoring system based on image recognition according to an embodiment of the present invention.

[0062] The present invention also provides a warehouse safety monitoring system based on image recognition. Figure 3 As shown, the system includes a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, a warehouse workshop safety monitoring method based on image recognition according to the first aspect of the present invention is implemented.

[0063] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present invention, and does not constitute a limitation on the computer device of the present invention. The specific computer device may include more or less components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0064] The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface, whose configuration and functions are known in the art and thus will not be described in detail here.

[0065] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, or any other medium that may be used to store the desired information and may be accessed by an application, module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0066] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0067] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A warehouse safety monitoring method based on image recognition, characterized in that: include: A neural network is used to obtain a first rectangle corresponding to the stone pile and a second rectangle corresponding to the dust and smoke in each frame of the storage workshop image; the stone pile feature of the first rectangle is determined according to the distribution of edge pixel points in the first rectangle; Determine the similarity of the stone stacks between different frames according to the similarity between the stone stack features corresponding to the first rectangle in the warehouse images of different frames; According to the similarity of the stone pile between the warehouse image at the current moment and the warehouse images of several previous frames and the change of the first rectangle, determine the first possibility of the collapse of the stone pile at the current moment, wherein the first possibility is negatively correlated with the similarity of the stone pile, and positively correlated with the area proportion of all the first rectangles in the warehouse image at the current moment and the difference between the areas of all the first rectangles in the warehouse image at the current moment and the warehouse images of several previous frames; Determine a second possibility of the collapse of the stone pile according to the second rectangle in the storage workshop image at the current moment and the first possibility, wherein the second possibility is positively correlated with the first possibility and the area ratio of all second rectangles in the storage workshop image at the current moment, and is positively correlated with the difference between the area of ​​the minimum circumscribed rectangle of all second rectangles and the area of ​​all second rectangles; In response to the second possibility of the rock pile collapsing being greater than the collapse threshold, a safety alarm is issued.

2. According to the method of claim 1, the method is characterized in that: The step of determining the first rectangular stone stacking feature comprises: Resize the image in the first rectangle to The fixed size of the first rectangle is used to detect the edge of the adjusted first rectangle to obtain an edge image; the stone stacking features of the first rectangle are determined according to the edge image. : , is the preset side length; , , They represent the edge images Row, No. Row, No. The cumulative sum of the grayscale values ​​of the pixels in the row; , , They represent the edge images Column, No. Column, No. The cumulative sum of the grayscale values ​​of the pixels in the column; , , They represent the edge images Row, No. Row, No. The gray value of the row is The mean of the horizontal coordinates of the pixels; , , They represent the edge images Column, No. Column, No. The gray value of the column is The mean of the vertical coordinates of the pixels.

3. The method for warehouse safety monitoring based on image recognition according to claim 1 is characterized in that: The determining of the similarity of the stone stacks between different frames comprises: The warehouse images of several frames before the current warehouse image are used as reference images respectively; the similarity of the stone stacking between the warehouse image at the current moment and each reference image is determined: , The image of the warehouse at the current moment and the Similarity of stone piles between reference images; Indicates the current moment of the warehouse image The first rectangular stone pile features, Take all Integers in Indicates the number of first rectangles in the warehouse workshop image at the current moment; The current warehouse image The first rectangle is in A first rectangular stone stack feature corresponding to a reference image; represents the cosine similarity function; Represents the minimum function.

4. A warehouse safety monitoring method based on image recognition according to claim 3, characterized in that: The first possibility of the stone pile collapse at the current moment satisfies the expression: ； in, Indicates the first possibility of the stone pile collapsing at the current moment; Represents the sum of the areas of all first rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; The image of the warehouse at the current moment and the The similarity of stone stacks between reference images, Take all Integers in represents a preset first quantity; Indicates The sum of the areas of all first rectangles in the reference images; represents the natural exponential function.

5. A warehouse safety monitoring method based on image recognition according to any one of claims 1-4, characterized in that: The second possibility of the stone pile collapse satisfies the expression: ； in, Indicates the second possibility of the collapse of the stone pile; Represents the sum of the areas of all second rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; Indicates the area of ​​the minimum circumscribed rectangle of all second rectangles in the warehouse workshop image at the current moment; Indicates the first possibility of the stone pile collapsing at the current moment.

6. A warehouse safety monitoring method based on image recognition according to any one of claims 1-4, characterized in that: The second possibility of the stone pile collapse satisfies the expression: ； in, It indicates the second possibility of the collapse of the stone pile, reflecting the possibility of the linkage collapse of the stone pile; Represents the sum of the areas of all second rectangles in the warehouse workshop image at the current moment; Indicates the area size of the warehouse workshop image; Indicates the area of ​​the minimum circumscribed rectangle of all second rectangles in the warehouse workshop image at the current moment; Indicates the wind speed at the current moment; represents a hyperparameter; Indicates the first possibility of the stone pile collapsing at the current moment; represents the natural exponential function.

7. A warehouse safety monitoring method based on image recognition according to any one of claims 1-4, characterized in that: Also includes: In response to the absence of the second rectangle in the storage workshop image at the current moment, if the first possibility of the stone pile collapse at the current moment is greater than the collapse threshold, a safety alarm is issued.

8. The method for warehouse safety monitoring based on image recognition according to claim 1 is characterized in that: The neural network adopts the YOLOv5 model.

9. A warehouse safety monitoring system based on image recognition, comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor executes the computer program to implement a warehouse workshop safety monitoring method based on image recognition as described in any one of claims 1-8.

Images