Method and device for monitoring environment in granary, equipment and storage medium

Abstract

The embodiment of the invention discloses an environment monitoring method and device in a granary, equipment and a storage medium. The method comprises the following steps: acquiring current image data of the surface of a grain pile acquired by image acquisition equipment in the granary; the current image data and a preset foreign matter recognition model are used for foreign matter recognition, a foreign matter recognition result is determined, and the foreign matter recognition result is used for reflecting whether foreign matter exists on the surface of the grain pile; the current image data and a preset optical flow algorithm are used for anomaly recognition, an anomaly recognition result is determined, and the anomaly recognition result is used for reflecting whether the surface of the grain pile is abnormal or not; and outputting a grain pile surface monitoring prompt according to the foreign matter recognition result and the abnormity recognition result. Through the mode, the granary can be supervised by using the image acquisition equipment, and abnormal and foreign matter identification on the surface of the grain pile is carried out on the image data, so that the granary supervision efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of grain storage management, and in particular to a method and device, equipment and storage medium for monitoring the environment in a grain storage. Background Art

[0002] Grain storage is an important part of ensuring food safety and quality. Traditional grain depot management relies on manual inspection and monitoring, but due to the complex environment (such as insufficient light, cramped space, etc.) and frequent operations of personnel in the grain depot, it is difficult to achieve comprehensive and accurate monitoring of foreign objects. The presence of foreign objects (such as leftover tools, dropped objects, people or other debris) in the grain depot may not only affect the quality of grain, but also the safety of storage operations. Therefore, an efficient grain depot supervision method is urgently needed. Summary of the invention

[0003] The main purpose of the present invention is to provide a method and device, equipment and storage medium for environmental monitoring in a granary, which can solve the problem of lack of efficient granary supervision means in the prior art.

[0004] To achieve the above object, the present invention provides a method for monitoring the environment in a granary in a first aspect, the method comprising: Acquire current image data of the grain pile surface acquired by the image acquisition device in the grain bin; Using the current image data and a preset foreign object recognition model to perform foreign object recognition, and determine a foreign object recognition result, wherein the foreign object recognition result is used to reflect whether there is a foreign object on the surface of the grain pile; Using the current image data and a preset optical flow algorithm to perform abnormality recognition, and determine an abnormality recognition result, wherein the abnormality recognition result is used to reflect whether there is an abnormality on the surface of the grain pile; Output monitoring prompts for the grain pile surface according to the foreign object identification results and the abnormality identification results.

[0005] In a feasible implementation, the foreign body recognition model includes at least a convolutional layer, a pooling layer, and a fully connected layer connected in sequence, and the foreign body recognition is performed using the current image data and the preset foreign body recognition model to determine the foreign body recognition result, including: Inputting the current image data into the convolution layer for convolution operation to obtain a first feature map; Inputting the first feature map into the pooling layer for pooling operation to obtain a target feature map; Inputting the target feature map into the fully connected layer to perform target classification, and obtaining a classification result, wherein the classification result at least includes a correspondence between a plurality of foreign object categories and a predicted probability of the foreign object category, and the predicted probability is used to reflect the prediction credibility of the foreign object category; A foreign object category with the largest predicted probability is selected from the classification results as the target category of the current image data, and the foreign object recognition result includes the target category.

[0006] In a feasible implementation, the current image data includes a current frame image and a previous frame image, and the abnormality recognition is performed using the current image data and a preset optical flow algorithm to determine the abnormality recognition result, including: Using the current frame image and the previous frame image to perform gradient calculation, a horizontal gradient, a vertical gradient and a time gradient of the current frame image are obtained; Using the horizontal gradient, vertical gradient, time gradient and preset optical flow algorithm, a motion vector of each pixel point of the current frame image is obtained; Performing optical flow visualization processing based on the motion vector to obtain an optical flow field map; The abnormality recognition result is determined according to the optical flow field map and the preset abnormality determination rules.

[0007] In a feasible implementation, determining the abnormality recognition result according to the optical flow field map and a preset abnormality determination rule includes: If the motion vector of the target area in the optical flow field image is not zero, and the area of ​​the target area is greater than a preset area threshold, the abnormal recognition result is determined to be a first result, and the first result is used to reflect that the target area on the surface of the grain pile has moved; If the motion vectors in the optical flow field map are all zero, the abnormal identification result is determined to be the second result, and the second result is used to reflect that there is no movement on the surface of the grain pile.

[0008] In a feasible implementation, the step of obtaining the current image data of the grain pile surface acquired by the image acquisition device in the grain bin may then include: The current image data is preprocessed to obtain preprocessed current image data, wherein the preprocessing at least includes denoising, image enhancement and binarization processing.

[0009] In a feasible implementation, the method then includes: Analyzing illumination distribution of the current image data; If the illumination distribution is uneven, the brightness and contrast of the current image data are adjusted based on the illumination distribution to perform illumination compensation, so as to obtain the current image data after illumination compensation.

[0010] In a feasible implementation, outputting a monitoring prompt of the grain pile surface according to the foreign object recognition result and the abnormality recognition result includes: Determining a current risk value of the grain pile surface according to the foreign object identification result, the abnormality identification result and a preset risk function; If the current risk value is greater than a preset risk threshold, a warning prompt is output, and the monitoring prompt includes the warning prompt.

[0011] To achieve the above object, the second aspect of the present invention provides an environment monitoring device in a granary, the device comprising: Data acquisition module: used to acquire current image data of the grain pile surface acquired by the image acquisition device in the grain bin; A foreign body recognition module is used to recognize foreign bodies by using the current image data and a preset foreign body recognition model, and to determine a foreign body recognition result, wherein the foreign body recognition result is used to reflect whether there are foreign bodies on the surface of the grain pile; Anomaly recognition module: used to use the current image data and a preset optical flow algorithm to perform anomaly recognition and determine anomaly recognition results, wherein the anomaly recognition results are used to reflect whether there is an abnormality on the surface of the grain pile; Prompt output module: used to output monitoring prompts of the grain pile surface according to the foreign object identification results and the abnormality identification results.

[0012] To achieve the above-mentioned purpose, the third aspect of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps shown in the first aspect and any feasible implementation method.

[0013] To achieve the above-mentioned purpose, the fourth aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps shown in the first aspect and any feasible implementation method.

[0014] The embodiments of the present invention have the following beneficial effects: The present invention provides an environmental monitoring method in a granary, the method comprising: obtaining current image data of the grain pile surface collected by an image acquisition device in the granary; using the current image data and a preset foreign object recognition model to perform foreign object recognition, and determining a foreign object recognition result, the foreign object recognition result is used to reflect whether there is a foreign object on the grain pile surface; using the current image data and a preset optical flow algorithm to perform abnormality recognition, and determining an abnormality recognition result, the abnormality recognition result is used to reflect whether there is an abnormality on the grain pile surface; outputting a monitoring prompt for the grain pile surface according to the foreign object recognition result and the abnormality recognition result. In the above manner, the image acquisition device can be used to monitor the granary, and the image data can be used to identify abnormalities and foreign objects on the grain pile surface, thereby improving the efficiency of granary supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] in: Figure 1 This is a flow chart of a method for monitoring the environment in a granary according to an embodiment of the present invention; Figure 2 is another flow chart of an environmental monitoring method in a granary according to an embodiment of the present invention; Figure 3 This is a structural block diagram of an environment monitoring device in a granary according to an embodiment of the present invention; Figure 4 4 is a structural block diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 , Figure 1 This is a flow chart of an environmental monitoring method in a granary in an embodiment of the present invention. The environmental monitoring method in a granary is applied to an environmental monitoring system in a granary. The environmental monitoring system in a granary includes a terminal and a server. The terminal and the server are connected via a network. The terminal can be a desktop terminal or a mobile terminal. The mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server can be implemented as an independent server or a server cluster composed of multiple servers. The method can be applied to both a terminal and a server. This embodiment is described by applying it to a terminal. Figure 1 The steps include: 101. Acquire current image data of the grain pile surface acquired by an image acquisition device in the grain bin; It should be noted that in order to improve the efficiency of supervision of the granary, the present application supervises the granary through video surveillance, by installing image acquisition equipment at multiple locations in the granary, collecting images of the surface of the grain pile through the image acquisition equipment, and using the collected images to automatically supervise the granary. The image acquisition equipment includes but is not limited to high-definition cameras or other imaging devices. High-definition cameras or other imaging devices can be deployed at multiple key locations in the granary to cover all areas in the granary where foreign objects may appear, ensuring that there are no blind spots, and collecting image data of the surface of the grain pile in real time. The collected image should have sufficient resolution and clarity to ensure that foreign objects can be clearly identified. In this way, the current image data of the surface of the grain pile collected by the image acquisition device in the granary is obtained, wherein the current image data can be an image sequence, and the image sequence includes one or more frames of images.

[0019] 102. Perform foreign object recognition using the current image data and a preset foreign object recognition model to determine a foreign object recognition result, wherein the foreign object recognition result is used to reflect whether there is a foreign object on the surface of the grain pile; 103. Perform abnormality recognition using the current image data and a preset optical flow algorithm to determine an abnormality recognition result, wherein the abnormality recognition result is used to reflect whether there is an abnormality on the surface of the grain pile; Furthermore, the foreign body recognition model is used to recognize foreign bodies in the current image data, and the optical flow algorithm is used to recognize anomalies in the current image data to obtain foreign body recognition results and anomaly recognition results.

[0020] Among them, the foreign body recognition result is used to reflect whether there are foreign bodies on the surface of the grain pile. The foreign body recognition result at least includes the foreign body category and may also include the probability of the foreign body category. The abnormal recognition result is used to reflect whether there is an abnormality on the surface of the grain pile, such as whether there is abnormal movement on the surface of the grain pile. Foreign body recognition is to identify the segmented target area through machine learning or deep learning models (such as convolutional neural networks (CNN). The model learns to distinguish between different types of foreign bodies (such as leftover tools, dropped objects, people, etc.) and grain in the grain pile through training data, so as to achieve accurate recognition. Abnormal detection is to use image data for further analysis and processing of abnormal conditions that may exist on the surface of the grain pile (such as large-scale movement of objects, uneven accumulation, etc.), and timely discover and feedback abnormal information.

[0021] Exemplarily, the foreign body recognition model can be a pre-trained machine learning model or deep learning model, which collects image data in different scenes, including images of grain piles, foreign objects, and normal grain storage environments, and uses these data to train a deep learning model (such as a convolutional neural network CNN) to improve the accuracy of recognition. For example, by pre-collecting images of the surface of the grain pile, marking the foreign object category label for each foreign object in the image, obtaining sample images with foreign object category labels, and using the sample images to train the original machine learning model or deep learning model. When the machine learning model or deep learning model converges, a foreign body recognition model is obtained. Among them, the optical flow method can analyze the movement of each pixel in the image, so as to realize the recognition of abnormal movement on the surface of the grain pile. For abnormal situations such as "large-area movement of objects and uneven accumulation of objects", the system can identify them through the optical flow method and motion detection algorithm. The optical flow method can detect the movement of objects in the image, and by calculating the motion vector (horizontal and vertical components) of the pixel points in the image, it can determine whether there is a large-scale movement or uneven accumulation on the surface of the grain pile.

[0022] 104. Output monitoring prompts for the grain pile surface according to the foreign matter identification result and the abnormality identification result.

[0023] Furthermore, in order to improve the accuracy of granary monitoring, monitoring prompts of the grain pile surface are output through the foreign object recognition results and the abnormal recognition results. The monitoring prompts are used to remind managers of the current status of the granary so that managers can know and respond in time.

[0024] For example, the identified targets are classified into categories such as "leftover tools", "dropped objects", "personnel" or "other items", and corresponding treatment measures are performed according to the classification results. For example, for leftover tools or dropped objects, the system can issue an alarm to notify warehouse staff to remove them; for personnel identification, the system can trigger a security alarm. Abnormal information will also be discovered and fed back in a timely manner. Among them, all identification and classification results will be recorded and stored in real time for later analysis and tracing. The system can provide a playback function for historical data for monitoring and management.

[0025] The present invention provides an environmental monitoring method in a granary, the method comprising: obtaining current image data of the grain pile surface collected by an image acquisition device in the granary; using the current image data and a preset foreign object recognition model to perform foreign object recognition, and determining a foreign object recognition result, the foreign object recognition result is used to reflect whether there is a foreign object on the grain pile surface; using the current image data and a preset optical flow algorithm to perform abnormality recognition, and determining an abnormality recognition result, the abnormality recognition result is used to reflect whether there is an abnormality on the grain pile surface; outputting a monitoring prompt for the grain pile surface according to the foreign object recognition result and the abnormality recognition result. In the above manner, the image acquisition device can be used to monitor the granary, and the image data can be used to identify abnormalities and foreign objects on the grain pile surface, thereby improving the efficiency of granary supervision.

[0026] See also Figure 2 , Figure 2 is another flow chart of a method for monitoring the environment in a granary according to an embodiment of the present invention, Figure 2 The steps include: 201. Acquire current image data of the grain pile surface acquired by an image acquisition device in the grain bin; It should be noted that step 201 and Figure 1 The contents of step 101 are similar, and are not described here to avoid repetition. For details, please refer to Figure 1 The content of step 101 is shown.

[0027] In a feasible implementation, the collected image data may contain noise or interference information, so the image needs to be preprocessed first, including denoising, image enhancement, illumination compensation, etc. Then, the image segmentation technology is used to extract the possible foreign matter area in the image to generate the target area, that is, after 201, it also includes: preprocessing the current image data to obtain the preprocessed current image data, and the preprocessing at least includes denoising, image enhancement and binarization processing.

[0028] For example, compared with general image processing scenarios, the granary environment has the following particularities: (1) Complex lighting conditions: The interior of a granary is usually poorly lit, and there may be uneven lighting distribution (such as local strong light or shadows), which places higher requirements on the accuracy of image acquisition and processing. (2) Background complexity: There may be areas with complex textures and similar colors on the surface of a grain pile, making it difficult to distinguish between foreign objects (such as tools, dropped objects, etc.) and grain. (3) Dynamic changes: The surface of a grain pile may change dynamically due to the flow of grain, the movement of accumulated objects, etc., requiring real-time detection and processing. (4) Diversity of foreign objects: There are many types of foreign objects in a granary (such as tools, people, dropped objects, etc.), and they vary in shape, size, and color, which increases the difficulty of identification. Therefore, the following algorithm is used for image preprocessing and target segmentation, and illumination compensation is also performed: 1) In the process of image data processing, the original image needs to be denoised and enhanced first to improve the accuracy of subsequent recognition. Denoising methods include Gaussian filtering and median filtering. This application takes Gaussian filtering as an example. The Gaussian filtering formula is as follows: ; Among them, I ( x , y ) is the original image pixel value, is the pixel value of the filtered image, is the standard deviation, which determines the range of the filter.

[0029] 2) Image enhancement uses methods such as histogram equalization to improve image contrast. The histogram equalization formula is as follows: ; in, is the grayscale of the image, is the pixel frequency of the original image, is the cumulative distribution function, which represents the cumulative probability of gray levels.

[0030] 3) Image segmentation uses a threshold-based segmentation method. Assuming that the pixel intensity range of the target area is known, a simple threshold operation is used to segment the target, as shown in the following binarization formula: ; in, is the preset threshold, is the pixel intensity of the image, is the binary image after segmentation.

[0031] In a feasible implementation, the method will then perform illumination compensation including: analyzing the illumination distribution of the current image data; if the illumination distribution is uneven, adjusting the brightness and contrast of the current image data based on the illumination distribution to perform illumination compensation, thereby obtaining the current image data after illumination compensation.

[0032] It is understandable that the image data used for subsequent recognition may be the original current image data or the high-quality current image data after preprocessing, and this application does not limit this.

[0033] 202. Perform foreign object recognition using the current image data and a preset foreign object recognition model to determine a foreign object recognition result, wherein the foreign object recognition result is used to reflect whether there is a foreign object on the surface of the grain pile; It should be noted that step 202 and Figure 1 The contents of step 102 are similar and will not be described here to avoid repetition. Figure 1 The content of step 102 is shown.

[0034] In a feasible implementation, the foreign body recognition model may be a convolutional neural network, the core idea of ​​which is to extract image features through a convolutional layer and perform classification through a fully connected layer. The foreign body recognition model at least includes a convolutional layer, a pooling layer, and a fully connected layer connected in sequence. The foreign body recognition is performed using the current image data and the preset foreign body recognition model to determine the foreign body recognition result, including A01 to A04: A01. Input the current image data into a convolution layer for convolution operation to obtain a first feature map; Exemplarily, the convolution operation formula is as follows: ; Among them, I is the input image, K is the convolution kernel, is the feature map after convolution, and k is the size of the convolution kernel.

[0035] A02. Input the first feature map into the pooling layer for pooling operation to obtain a target feature map; Exemplarily, the pooling operation is used to reduce the size of the feature map, and the pooling method is maximum pooling, as shown in the following formula: ; Among them, poolregion is the pooling area, is the result after pooling, that is, the target feature map.

[0036] A03, inputting the target feature map into the fully connected layer to perform target classification, and obtaining a classification result, wherein the classification result at least includes a correspondence between a plurality of foreign body categories and the predicted probabilities of the foreign body categories, and the predicted probabilities are used to reflect the prediction credibility of the foreign body categories; For example, after several convolutional and pooling layers, the output feature map will be passed through a fully connected layer for object classification, using a softmax activation function: ; in, is the probability of the classification result, W k and are the weights and biases of the classification layer, x are input features, j represents all possible categories.

[0037] Taking the current image data after illumination compensation as an example, the current image data is input into the convolution layer for convolution operation to obtain the first feature map. The first feature map is then input into the pooling layer for pooling operation to obtain the target feature map. Finally, the target is classified through the fully connected layer to obtain the classification result, wherein the classification result at least includes the correspondence between multiple foreign body categories and the predicted probability of the foreign body category, the predicted probability is used to reflect the prediction credibility of the foreign body category, and the foreign body recognition result may include the classification result.

[0038] A04. Select the foreign body category with the largest predicted probability from the classification results as the target category of the current image data, and the foreign body recognition result includes the target category.

[0039] Furthermore, the present application adopts, to obtain the final classification result, to obtain the foreign body recognition result. Specifically, the foreign body category with the largest predicted probability is selected from the classification result as the target category of the current image data, and the foreign body recognition result includes the target category.

[0040] For example, according to the output of the convolutional neural network (CNN), the target will be classified into categories such as "leftover tools", "dropped objects", "personnel", etc. Assuming that the model output is the probability of each category, the final classification result can be determined by the maximum method: ; in, is the predicted category, is the predicted probability for each category.

[0041] Furthermore, the following foreign body classification rules can be combined to comprehensively judge foreign bodies: Legacy tools: A convolutional neural network (CNN) is used to identify objects with regular shapes and colors that are significantly different from the grain pile (such as shovels, brooms, etc.), and their location information (such as whether they are located on the surface of the grain pile) is used for judgment.

[0042] Falling objects: Identify objects with irregular shapes and colors similar to those of grain piles (such as packaging bags, ropes, etc.), and make judgments based on their movement status (such as whether they have fallen from a height).

[0043] Personnel: Personnel are identified through human detection algorithms (such as YOLO, OpenPose, etc.), and judged based on their behavioral characteristics (such as whether they are active in a dangerous area).

[0044] Other items: For objects that cannot be clearly classified, the system will mark them as "other items" and record their location and characteristics for further inspection by staff.

[0045] Exemplary rules for distinguishing the motion state of foreign matter: Stationary foreign objects: The motion state of the object is detected by the optical flow method. If the optical flow vector of the object in consecutive frames is zero or close to zero, it is judged as a stationary foreign object.

[0046] Moving foreign objects: If the optical flow vector of an object in consecutive frames is not zero, it is considered a moving foreign object. The system will further analyze its motion trajectory to determine whether it is abnormal movement (such as large-area movement).

[0047] 203. Perform abnormality recognition using the current image data and a preset optical flow algorithm to determine an abnormality recognition result, where the abnormality recognition result is used to reflect whether there is an abnormality on the surface of the grain pile; It should be noted that step 203 and Figure 1 The contents of step 103 are similar, and are not described here to avoid repetition. For details, please refer to Figure 1 The content of step 103 is shown.

[0048] In a feasible implementation, in order to ensure the safety in the grain warehouse, the system also needs to detect potential abnormal situations in real time. For example, uneven or large-scale movement of objects on the surface of the grain pile may indicate loose piles or other potential risks. Use an image-based motion detection algorithm to achieve this function. The motion detection algorithm includes but is not limited to the optical flow method, wherein the current image data includes the current frame image and the previous frame image, and the abnormality recognition is performed using the current image data and the preset optical flow algorithm to determine the abnormality recognition result, including steps B01 to B04: B01, performing gradient calculation using the current frame image and the previous frame image to obtain the horizontal gradient, vertical gradient and time gradient of the current frame image; B02, using the horizontal gradient, vertical gradient, time gradient and a preset optical flow algorithm to obtain a motion vector of each pixel of the current frame image; B03, performing optical flow visualization processing based on the motion vector to obtain an optical flow field map; B04. Determine an abnormality recognition result according to the optical flow field map and preset abnormality determination rules.

[0049] It should be noted that by calculating the horizontal gradient, vertical gradient of the current frame image and the time gradient between the continuous frame images, the motion vector of each pixel point of the current frame image is obtained by utilizing the horizontal gradient, vertical gradient, time gradient and a preset optical flow algorithm. The motion vector is also called optical flow, which can represent the size and direction of the movement of the pixel point. An optical flow field map is constructed by the motion vector of each pixel point on the image. Through this map, you can intuitively see where there is movement and where there is movement, thereby obtaining abnormal recognition results.

[0050] Exemplarily, the optical flow method is used to detect the motion of objects in an image, and its basic formula is: ; in, , are the horizontal and vertical components of the optical flow, representing the speed of movement of pixels in the image in the x and y directions. , , are the gradient of the image in the x and y directions and the change in time t, , They are the horizontal gradient and vertical gradient, i.e., the grayscale change rate of the image in the horizontal and vertical directions; It is the rate of change of the image in the time direction, expressed as the time gradient, obtained through two consecutive frames of images.

[0051] After obtaining the gradient, the optical flow components u and v are solved according to the basic formula of the optical flow method.

[0052] Solution method: Since the optical flow equation is an underdetermined equation (one equation, two unknowns), it is usually necessary to introduce additional constraints to solve it. Common methods include: Lucas-Kanade method: Assuming that the optical flow in the local area is the same, u and v are solved by the least squares method; Horn-Schunck method: Introduce global smoothness constraints and solve u and v through optimization algorithms.

[0053] In a feasible implementation, the abnormal displacement detected by the optical flow method can trigger a safety warning of the system to notify the staff to check the grain pile. Step B04 includes steps C01 to C02: C01. If the motion vector of the target area in the optical flow field image is not zero, and the area of ​​the target area is greater than a preset area threshold, the abnormal recognition result is determined to be a first result, and the first result is used to reflect that the target area on the surface of the grain pile is moving; C02. If the motion vectors in the optical flow field map are all zero, the abnormality recognition result is determined to be a second result, and the second result is used to reflect that there is no movement on the surface of the grain pile.

[0054] It can be understood that the present application considers that when the motion vector is 0, there is no motion, and therefore, there is no movement. Therefore, if the motion vector is not zero, it is considered that the pixel has moved, and if there is a target area that is not zero, and the target area is larger than a preset area threshold, it can be considered that it has moved, wherein the area threshold may include a first area threshold and a second area threshold, and the first area threshold is smaller than the second area threshold. If the target area is larger than the first threshold and smaller than the second area threshold, it is considered that the target area belongs to a small area or local movement, which may be caused by uneven grain or movement of foreign objects other than grain; if the target area is larger than the second area threshold, it is considered that the target area belongs to a large area movement, which may be a large area movement of grain, and there is a possibility of collapse. Furthermore, the abnormal recognition results may include movement or no movement, and the type of movement, which can be specifically divided into foreign object movement, uneven grain stacking, large area movement of grain, etc.

[0055] For example, the uniformity of foreign matter distribution can be further analyzed by density, and the judgment rules are as follows: Uniform distribution: The grain pile surface is divided into multiple areas through image segmentation technology, and the density of foreign matter in each area is calculated. If the density of foreign matter in each area is similar, it is judged to be uniformly distributed.

[0056] Uneven distribution: If the density of foreign matter in some areas is significantly higher than in other areas, it is considered uneven distribution. The system will record the location of the uneven area and trigger an alarm.

[0057] In addition, it can also be combined with motion trajectory analysis: for detected moving objects, the system will record their motion trajectories and analyze their motion patterns. For example: If the object's movement appears random, this may indicate it was dropped.

[0058] If the object's movement pattern appears regular, it may indicate that it is a person or mechanical equipment.

[0059] 204. Determine a current risk value of the grain pile surface according to the foreign matter identification result, the abnormality identification result and a preset risk function; 205. If the current risk value is greater than a preset risk threshold, a warning prompt is output, and the monitoring prompt includes the warning prompt.

[0060] Furthermore, the risk of the existence of the grain pile surface can be evaluated by using the foreign body identification result and the abnormality identification result, and the current risk value of the grain pile surface can be determined according to the foreign body identification result, the abnormality identification result and the preset risk function. If the current risk value is greater than the preset risk threshold, an early warning prompt is output, and the monitoring prompt includes the early warning prompt.

[0061] For example, based on the target recognition, classification and anomaly detection results, the system will make a comprehensive judgment based on the following rules to determine whether an alarm or other processing measures need to be triggered: ; in, , , are the classification scores for tools, dropped objects, and personnel, respectively. is the anomaly detection score, , , , is the weight coefficient, S is the current risk value. If S>threshold, an alarm is triggered.

[0062] By assigning different weights, different prompts can be output for different processing. For example, for each foreign body category, corresponding processing can be performed: Left-behind tools: If a tool is identified as left-behind, the system will sound an alarm and notify the staff to clean it up.

[0063] Dropped objects: When a dropped object is identified, the system will record its location and monitor it in real time.

[0064] People: When a person is detected, the system triggers a safety alarm to ensure that the person is not in the danger zone.

[0065] The present invention provides an environmental monitoring method in a granary. The above method has the following beneficial effects: 1. Efficient foreign body recognition capability: The present invention can efficiently recognize different types of foreign bodies, including abandoned tools, dropped objects, personnel and other sundries, through a deep learning algorithm, and improve the automation level of grain depot management. 2. Automation and real-time performance: By using cameras and image processing technology, the system can collect data, process information and output feedback in real time, greatly reducing the workload of manual inspection and improving work efficiency and response speed. 3. Accurate target classification: Through the training of a deep learning model, it can accurately distinguish between targets and foreign bodies in a grain depot, avoiding the problems of misjudgment and missed judgment in traditional methods. 4. Reduction of labor costs and safety hazards: Automated recognition and processing not only reduces labor costs, but also improves the safety of grain depot operations, and prevents potential threats of foreign bodies to grain storage quality and operational safety. 5. Scalability: The technical solution of the present invention is not only applicable to foreign body recognition in grain depots, but can also be extended to foreign body detection and processing in other similar environments (such as warehouses, production lines, etc.).

[0066] See also Figure 3 , Figure 3 FIG. 1 is a structural block diagram of an environment monitoring device in a granary according to an embodiment of the present invention. Figure 3 The device shown comprises: Data acquisition module 301: used to acquire current image data of the grain pile surface acquired by the image acquisition device in the grain bin; The foreign object recognition module 302 is used to use the current image data and a preset foreign object recognition model to perform foreign object recognition and determine a foreign object recognition result, wherein the foreign object recognition result is used to reflect whether there is a foreign object on the surface of the grain pile; Anomaly recognition module 303: used to perform anomaly recognition using the current image data and a preset optical flow algorithm to determine anomaly recognition results, wherein the anomaly recognition results are used to reflect whether there is an abnormality on the surface of the grain pile; The prompt output module 304 is used to output a monitoring prompt of the grain pile surface according to the foreign matter recognition result and the abnormality recognition result.

[0067] It should be noted that if Figure 3 The functions of each module in the device shown are Figure 1 The contents of each step are similar, so we will not repeat them here to avoid repetition. For details, please refer to Figure 1 The contents of each step in .

[0068] The present invention provides an environmental monitoring device in a granary, the device comprising: a data acquisition module: used to acquire current image data of the grain pile surface collected by an image acquisition device in the granary; a foreign object recognition module: used to use the current image data and a preset foreign object recognition model to perform foreign object recognition, determine the foreign object recognition result, and the foreign object recognition result is used to reflect whether there are foreign objects on the grain pile surface; an abnormality recognition module: used to use the current image data and a preset optical flow algorithm to perform abnormality recognition, determine the abnormality recognition result, and the abnormality recognition result is used to reflect whether there are abnormalities on the grain pile surface; a prompt output module: used to output monitoring prompts of the grain pile surface according to the foreign object recognition result and the abnormality recognition result. Through the above method, the image acquisition device can be used to monitor the granary, and the image data can be used to identify abnormalities and foreign objects on the grain pile surface, thereby improving the efficiency of granary supervision.

[0069] Figure 4 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. Those skilled in the art can understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0070] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Figure 1 or Figure 2 Steps shown.

[0071] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor performs the following steps: Figure 1 or Figure 2 Steps shown.

[0072] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0073] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for monitoring the environment in a granary, characterized in that: The method comprises: Acquire current image data of the grain pile surface acquired by the image acquisition device in the grain bin; Using the current image data and a preset foreign object recognition model to perform foreign object recognition, and determine a foreign object recognition result, wherein the foreign object recognition result is used to reflect whether there is a foreign object on the surface of the grain pile; Using the current image data and a preset optical flow algorithm to perform abnormality recognition, and determining an abnormality recognition result, wherein the abnormality recognition result is used to reflect whether there is an abnormality on the surface of the grain pile; Output monitoring prompts for the grain pile surface according to the foreign object identification results and the abnormality identification results.

2. The method according to claim 1, characterized in that: The foreign body recognition model at least includes a convolutional layer, a pooling layer and a fully connected layer connected in sequence, and the foreign body recognition is performed using the current image data and the preset foreign body recognition model to determine the foreign body recognition result, including: Inputting the current image data into the convolution layer for convolution operation to obtain a first feature map; Inputting the first feature map into the pooling layer for pooling operation to obtain a target feature map; Inputting the target feature map into the fully connected layer to perform target classification, and obtaining a classification result, wherein the classification result at least includes a correspondence between a plurality of foreign object categories and a predicted probability of the foreign object category, and the predicted probability is used to reflect the prediction credibility of the foreign object category; A foreign object category with the largest predicted probability is selected from the classification results as the target category of the current image data, and the foreign object recognition result includes the target category.

3. The method according to claim 1, characterized in that: The current image data includes a current frame image and a previous frame image, and the abnormality recognition is performed using the current image data and a preset optical flow algorithm to determine the abnormality recognition result, including: Using the current frame image and the previous frame image to perform gradient calculation, a horizontal gradient, a vertical gradient and a time gradient of the current frame image are obtained; Using the horizontal gradient, vertical gradient, time gradient and preset optical flow algorithm, a motion vector of each pixel point of the current frame image is obtained; Performing optical flow visualization processing based on the motion vector to obtain an optical flow field map; The abnormality recognition result is determined according to the optical flow field map and the preset abnormality determination rules.

4. The method according to claim 3, characterized in that: Determining the abnormality recognition result according to the optical flow field map and the preset abnormality determination rule includes: If the motion vector of the target area in the optical flow field image is not zero, and the area of ​​the target area is greater than a preset area threshold, the abnormal recognition result is determined to be a first result, and the first result is used to reflect that the target area on the surface of the grain pile has moved; If the motion vectors in the optical flow field map are all zero, the abnormal identification result is determined to be the second result, and the second result is used to reflect that there is no movement on the surface of the grain pile.

5. The method according to claim 1, characterized in that: The step of obtaining the current image data of the grain pile surface acquired by the image acquisition device in the grain bin further comprises: The current image data is preprocessed to obtain preprocessed current image data, wherein the preprocessing at least includes denoising, image enhancement and binarization processing.

6. The method according to claim 5, characterized in that: The method then comprises: Analyzing illumination distribution of the current image data; If the illumination distribution is uneven, the brightness and contrast of the current image data are adjusted based on the illumination distribution to perform illumination compensation, so as to obtain the current image data after illumination compensation.

7. The method according to claim 1, characterized in that: The outputting of the monitoring prompt of the grain pile surface according to the foreign object recognition result and the abnormality recognition result includes: Determining a current risk value of the grain pile surface according to the foreign object identification result, the abnormality identification result and a preset risk function; If the current risk value is greater than a preset risk threshold, a warning prompt is output, and the monitoring prompt includes the warning prompt.

8. An environmental monitoring device in a granary, characterized in that: The device comprises: Data acquisition module: used to acquire current image data of the grain pile surface acquired by the image acquisition device in the grain bin; A foreign body recognition module is used to recognize foreign bodies by using the current image data and a preset foreign body recognition model, and to determine a foreign body recognition result, wherein the foreign body recognition result is used to reflect whether there are foreign bodies on the surface of the grain pile; Anomaly recognition module: used to use the current image data and a preset optical flow algorithm to perform anomaly recognition and determine anomaly recognition results, wherein the anomaly recognition results are used to reflect whether there is an abnormality on the surface of the grain pile; Prompt output module: used to output monitoring prompts of the grain pile surface according to the foreign object identification results and the abnormality identification results.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images