Construction site scene detection method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a construction site scene detection method, apparatus, device and storage medium, wherein the method comprises: acquiring an image to be detected, wherein the image to be detected includes a construction site scene area; in the image to be detected, detecting construction site dust in the construction site scene area; in response to detecting the construction site dust, detecting construction site soil in the construction site scene area in at least one of the following frames of images: the image to be detected, and at least one frame of image that is temporally adjacent to the image to be detected.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and relate to but are not limited to a construction site scene detection method, device, equipment and storage medium. Background Art

[0002] With the development of urban infrastructure, the safety of construction sites in urban scenes is an important part of urban safety management. Although the new intelligent urban construction site scene early warning system can greatly reduce labor costs and achieve efficient management around the clock, it cannot guarantee the accuracy of construction site scene detection in the process of managing construction site scenes in urban scenes. Summary of the invention

[0003] The embodiment of the present application provides a technical solution for construction site scene detection.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a construction site scene detection method, the method comprising:

[0006] Acquire an image to be detected, wherein the image to be detected includes a construction site scene area;

[0007] In the image to be detected, detecting construction site dust in the construction site scene area;

[0008] In response to detecting the construction site dust, construction site soil in the construction site scene area is detected in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected.

[0009] In some embodiments, in response to detecting the construction site dust, the construction site soil in the construction site scene area is detected in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected, including: in response to detecting the construction site dust, the construction site soil in the construction site scene area is detected in the image to be detected; in response to not detecting the construction site soil in the image to be detected, at least one frame of image temporally adjacent to the image to be detected is determined; in the at least one adjacent frame of image, the construction site soil in the construction site scene area is detected. In this way, the accuracy of detecting the construction site soil in the construction site scene area is improved.

[0010] In some embodiments, the detecting of construction site dust in the construction site scene area in the image to be detected includes: extracting features of the image to be detected to obtain image features; detecting target image features matching preset dust features in the image features; and determining that the construction site dust is detected in the construction site scene area in response to detecting the target image features. In this way, by comparing the image features of the image to be detected, the construction site dust in the construction site scene can be detected more accurately.

[0011] In some embodiments, in response to detecting the construction site dust, the construction site soil in the construction site scene area is detected in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected, the method further includes: in response to detecting the construction site soil, using a semantic segmentation network to perform scene segmentation on the construction site soil and the construction site dust in the construction site scene area to obtain a segmentation result; wherein the semantic segmentation network is obtained by training a sample image set in which different categories of construction site dust and different categories of construction site soil are annotated in the picture; based on the segmentation result, the category of the construction site soil and the category of the construction site dust are determined; based on the category of the construction site soil and the category of the construction site dust, at least the construction site soil in the construction site scene area is processed. In this way, the categories of construction site dust and construction site soil in the construction site scene area can be determined more accurately, so as to facilitate targeted processing of construction site dust and construction site soil.

[0012] In some embodiments, the processing of at least the construction site soil in the construction site scene area based on the category of the construction site soil and the category of the construction site dust includes: determining the association between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust; when the association indicates that the construction site dust is related to the construction site soil, generating a first alarm message based on the category of the construction site soil to prompt the processing of the construction site soil. In this way, when there is an association between the construction site soil and the construction site dust, the alarm information of the construction site soil category is uploaded to the construction site management system so that the management personnel can process the construction site soil area in a timely manner.

[0013] In some embodiments, the determining the association between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust includes: determining the target category of the dust generated by the construction site soil of the category based on the category of the construction site soil; and determining the association based on the target category and the category of the construction site dust. In this way, by analyzing the category of construction site dust that can be generated by the construction site soil, it is possible to accurately determine whether the construction site dust comes from the construction site soil, thereby facilitating the subsequent determination of a more appropriate processing method for the construction site scene area.

[0014] In some embodiments, the determining of the association relationship based on the target category and the category of the construction site dust includes: if the target category includes the category of the construction site dust, determining that the construction site dust is related to the construction site soil; if the target category does not include the category of the construction site dust, determining that the construction site dust is not related to the construction site soil. In this way, by analyzing the similarity between the target category of dust generated by construction site soil and the category of construction site dust in the image, it is possible to more clearly and accurately determine whether the construction site dust and soil are associated.

[0015] In some embodiments, when the association relationship indicates that the construction site dust is related to the construction site soil, a first alarm message is generated based on the category of the construction site soil to prompt the construction site soil to be processed, including: when the association relationship indicates that the construction site dust is related to the construction site soil, based on the segmentation result, the first position information of the construction site soil is determined; based on the first position information and the category of the construction site soil, the first alarm message is generated to prompt the construction site soil to be processed based on the first position information in a manner that matches the category of the construction site soil. In this way, based on the location and category of the construction site soil, an alarm message matching the category is generated to facilitate the management personnel to manage the construction site soil covered by the location at that location.

[0016] In some embodiments, after determining the association relationship between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust, the method further includes: determining the first position information of the construction site soil and the second position information of the construction site dust based on the segmentation result when the association relationship indicates that the construction site dust is not related to the construction site soil; and generating a second alarm message based on the first position information, the category of the construction site soil, the second position information and the category of the construction site dust to prompt the construction site soil and the construction site dust to be processed respectively. In this way, based on the location and category of the construction site soil and the location and category of the construction site dust, the second alarm message is generated so that the management personnel can manage the construction site soil and the construction site dust more accurately.

[0017] In some embodiments, after detecting construction site dust in the construction site scene area in the image to be detected, the method further includes: detecting construction site soil in the construction site scene area in the image to be detected when the construction site dust is not detected; and generating first alarm information in response to detecting the construction site soil in the image to be detected based on first position information of the construction site soil in the image to be detected and the category of the construction site soil.

[0018] In some embodiments, in response to detecting the construction site dust, the construction site soil in the construction site scene area is detected in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected, the method further includes: in response to detecting the construction site dust and not detecting the construction site soil, generating a third alarm message based on the second position information of the construction site dust and the category of the construction site dust to prompt the use of a method that matches the category of the construction site dust and based on the second position information to process the construction site dust. In this way, different alarm messages are generated based on different detected contents, so that the actual situation of the construction site scene area can be fed back to the management personnel in a timely manner.

[0019] The present application provides a construction site scene detection device, the device comprising:

[0020] A first acquisition module, used to acquire an image to be detected, wherein the image to be detected includes a construction site scene area;

[0021] A first detection module, used to detect construction site dust in the construction site scene area in the image to be detected;

[0022] The second detection module is used to detect the construction site soil in the construction site scene area in at least one of the following frames of images in response to detecting the construction site dust: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected.

[0023] Correspondingly, an embodiment of the present application provides a computer storage medium, on which computer executable instructions are stored. After the computer executable instructions are executed, the above-mentioned construction site scene detection method can be implemented.

[0024] An embodiment of the present application provides a computer device, which includes a memory and a processor. The memory stores computer executable instructions, and the processor can implement the above-mentioned construction site scene detection method when running the computer executable instructions on the memory.

[0025] The embodiments of the present application provide a construction site scene detection method, apparatus, device and storage medium. For an image to be detected including a construction site scene area, construction site dust detection is first performed on the construction site scene area in the image to be detected; then, if construction site dust can be detected, construction site soil in the construction site scene area is continuously detected in the image to be detected or in an image temporally adjacent to the image to be detected; in this way, the construction site dust and construction site soil in the construction site scene area are detected in two stages, which can not only improve the extensiveness of detection, but also improve the accuracy of detection of construction site dust and construction site soil in the construction site scene area. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of the implementation flow of the construction site scene detection method provided in the embodiment of the present application;

[0027] Figure 2 Another implementation process diagram of the construction site scene detection method provided in the embodiment of the present application;

[0028] Figure 3 A schematic diagram of another implementation flow of the construction site scene detection method provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of an application scenario of the construction site scene detection method provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of another application scenario of the construction site scene detection method provided in an embodiment of the present application;

[0031] Figure 6 This is a schematic diagram of the structure of the construction site scene detection device according to an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the composition structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0034] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0035] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0037] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0038] 1) Semantic segmentation: Segmenting various types of information in an image. For example, marking a person’s outline in red and a road in purple. However, it is impossible to distinguish different people. This is equivalent to matching the external outlines of large categories of images with labels.

[0039] 2) Residual Neural Network (ResNet), which is built by residual modules, uses the gradient descent algorithm to train a neural network. As the number of layers increases, the training error decreases. This method can reach deeper layers of the network, help solve the problems of gradient vanishing and gradient exploding, and allow training of deeper networks while ensuring good performance.

[0040] The following describes an exemplary application of the device for detecting construction site scenes provided by the embodiments of the present application. The device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptop computers, tablet computers, desktop computers, cameras, mobile devices (e.g., personal digital assistants, dedicated messaging devices, portable gaming devices) with image acquisition functions, and can also be implemented as a server. The following describes an exemplary application when the device is implemented as a terminal or a server.

[0041] The method can be applied to a computer device, and the functions implemented by the method can be implemented by calling a program code through a processor in the computer device. Of course, the program code can be stored in a computer storage medium. It can be seen that the computer device at least includes a processor and a storage medium.

[0042] The present application embodiment provides a construction site scene detection method, such as Figure 1 As shown, combined with Figure 1 The following steps are explained:

[0043] Step S101, obtaining an image to be detected.

[0044] In some embodiments, the image to be detected includes a construction site scene area. The image to be detected may be an image with a complex appearance including the construction site scene area, or may be an image with a simple appearance including the construction site scene area. The image to be detected may be an image collected by any acquisition device in an urban scene, for example, the image content includes an image of piled-up block-shaped construction site soil on a wide road, or the image content includes an image of piled-up loose construction site soil, or the image content includes any one or more types of images such as dust, flying dust material stacking, or quasi-dust.

[0045] Step S102: detecting construction site dust in the construction site scene area in the image to be detected.

[0046] In some embodiments, after acquiring the image to be detected, firstly, whether there is construction site dust in the construction site scene area in the image to be detected. The construction site dust includes: dust and flying dust, etc. For example, a neural network is used to detect the construction site scene area in the image to be detected, and the construction site scene area is marked in the form of a detection frame; or, the construction site scene area is cut out from the image to be detected by cutting out the image, so as to detect the construction site dust in the construction site scene area.

[0047] In some possible implementations, a convolutional neural network or a semantic segmentation network is used to extract features from the image to be detected, and by identifying the extracted features, it is determined whether there is construction site dust in the construction site scene area, that is, the above step S102 can be implemented by the following steps S121 to S123 (not shown in the figure):

[0048] Step S121, extracting features from the image to be detected to obtain image features.

[0049] A convolutional neural network or a residual neural network is used to extract features from the image to be detected to obtain image features. The image features can be features corresponding to the construction site scene area or features of the entire image to be detected.

[0050] Step S122: Detect target image features that match preset dust features in the image features.

[0051] Here, the preset dust feature is obtained by extracting features from a real dust image, and the image features of the image to be detected are compared with the preset dust features. If there is a target image feature in the image features that has a high similarity with the preset dust feature, it is determined that the target image feature matches the preset dust feature.

[0052] Step S123, in response to detecting the target image feature, determining that the construction site dust is detected in the construction site scene area.

[0053] If the target image feature is detected in the image feature of the image to be detected, it means that there is a dust image area in the construction site scene area of ​​the image to be detected, which further means that construction site dust is detected in the construction site scene area. In this way, by comparing the image features of the image to be detected, construction site dust in the construction site scene can be detected more accurately.

[0054] Step S103 , in response to detecting the construction site dust, detecting the construction site soil in the construction site scene area in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected.

[0055] In some embodiments, the construction site soil includes loose soil and block soil, etc. If construction site dust is detected in the construction site scene area of ​​the image to be detected, the construction site soil detection can be continued in the construction site scene area of ​​the image to be detected, or the construction site soil detection can be performed in an image that is temporally adjacent to the image to be detected, so as to improve the detection accuracy.

[0056] In some possible implementations, in order to improve the accuracy of detecting the construction site soil in the construction site scene area, the above step S103 can be performed by Figure 2 The steps shown are implemented, Figure 2 Another implementation process diagram of the construction site scene detection method provided in the embodiment of the present application is shown in FIG. Figure 1 and 2 The following instructions are given:

[0057] Step S201 , in response to detecting the construction site dust, detecting the construction site soil in the construction site scene area in the image to be detected.

[0058] If construction site dust is detected in the construction site scene area of ​​the image to be detected, the detection priority of the image to be detected and the image adjacent to the image to be detected in time is determined as follows: the image to be detected is in front, and the image adjacent in time is in the back. That is, if construction site dust is detected in the construction site scene area of ​​the image to be detected, construction site soil is continuously detected in the construction site scene area of ​​the image to be detected.

[0059] In some possible implementations, the detection of construction site soil in the construction site scene area can be achieved through the above steps S121 to S123.

[0060] Step S202: in response to the construction site soil not being detected in the image to be detected, determining at least one frame of image that is temporally adjacent to the image to be detected.

[0061] If the construction site soil is not detected in the construction site scene area of ​​the image to be detected, that is, the construction site dust can be detected but the construction site soil cannot be detected in the construction site scene area of ​​the image to be detected, it means that the construction site soil may not be collected when collecting this frame of the image to be detected; in order to improve the accuracy of the construction site soil detection in the construction site scene area, at least one frame of image that is temporally adjacent to the image to be detected is obtained. For example, the previous frame of the image to be detected or the next frame of the image to be detected, etc. In other embodiments, the number of at least one frame of image that is temporally adjacent can be set, for example, the previous frame of the image to be detected and the previous frame of the previous frame are obtained, etc., to meet the preset number of images.

[0062] Step S203: detecting the construction site soil in the construction site scene area in the at least one adjacent frame of image.

[0063] In the acquired image that is temporally adjacent to the image to be detected, the construction site soil in the construction site scene area of ​​the adjacent image is detected. For example, it is set to acquire two frames of images that are temporally adjacent to the image to be detected (which can be the previous frame and the next frame of the image to be detected), and the construction site soil in the construction site scene area is detected in these two frames of images respectively; or, the construction site soil in the construction site scene area is detected in these two frames of images one by one; if the construction site soil can be detected in the first frame of image (the first frame of image can be any one of the two frames of image), the detection of the other frame of image is stopped. In this way, by detecting the construction site soil in at least one frame of image adjacent to the image to be detected, the accuracy of detecting the construction site soil and construction site dust in the construction site scene area can be improved.

[0064] In an embodiment of the present application, for an image to be detected whose screen includes a construction site scene area, construction site dust detection is first performed on the construction site scene area in the image to be detected; then, if construction site dust can be detected, construction site soil in the construction site scene area is continuously detected in the image to be detected or in an image temporally adjacent to the image to be detected; in this way, the detection of construction site dust and construction site soil in the construction site scene area is divided into two stages, which can not only improve the extensiveness of detection, but also improve the accuracy of detection of construction site dust and construction site soil in the construction site scene area.

[0065] In some embodiments, after the construction site soil and construction site dust in the construction site scene area are detected, the construction site scene area is processed according to the types of the construction site soil and construction site dust through the following steps S151 to 153 (not shown):

[0066] Step S151, in response to detecting the construction site soil, a semantic segmentation network is used to perform scene segmentation on the construction site soil and the construction site dust in the construction site scene area to obtain a segmentation result.

[0067] In some embodiments, the semantic segmentation network is obtained by training a sample image set with different categories of construction site soil labeled in the picture. The semantic segmentation network is obtained by using a model with strong robustness to guide the training of a segmentation network with a simple structure to be trained, so that the semantic segmentation network has high performance while maintaining low complexity.

[0068] The semantic segmentation network extracts features from the input image and performs semantic segmentation on the extracted features to generate a segmentation map that segments different categories of construction site soil and construction site dust in the image.

[0069] Each sample image in the sample image set has been labeled with different types of construction site soil and construction site dust in the picture using the preset semantic label library. Taking the different types of construction site soil as block construction site soil and scattered construction site soil, and the construction site dust as dust and flying dust as examples, the sample image set includes images of block construction site soil, scattered construction site soil, dust or flying dust in urban scenes. The preset semantic labels in the preset semantic label library are set according to the processing requirements carried in the image to be detected. For example, the demand to be processed in the image to be detected is the demand for monitoring the piled area of ​​block and scattered construction site soil in the urban scene. In the urban scene, the demand for this monitoring category includes: supervising the piled location, piled area and piled thickness of the construction site soil on the construction site, etc. Then the target category of the construction site soil to achieve this demand is the piled block construction site soil and scattered construction site soil, and the semantic label of block construction site soil is set to 1 and the semantic label of scattered construction site soil is set to 2; then in the sample image, the area where the scattered construction site soil is piled and the area where the block construction site soil is piled are marked as 1 and 2 respectively. In this way, the area where the construction site soil is piled in the sample image is marked by these preset semantic labels, and a sample image set that can be used as a training set is obtained to train the neural network to obtain a semantic segmentation network.

[0070] The process of using the semantic segmentation network to segment the construction site soil and construction site dust in the construction site scene area is as follows:

[0071] The semantic segmentation network includes: a residual neural network for feature extraction, a Deeplab v3 network for semantic segmentation of the extracted features, an upsampling module for upsampling the semantic segmentation results, and a final result prediction module.

[0072] In some possible implementations, first, the image to be detected is input into the residual neural network in the semantic segmentation network, and the image to be detected is subjected to feature extraction based on the residual neural network to obtain a feature map of the image to be detected, i.e., image features. The image to be detected whose content includes the construction site scene area is input into the residual neural network of the semantic segmentation network to realize feature extraction of the image to be detected. Then, based on the image features, the classified construction site soil is segmented to obtain a segmentation result. After feature extraction of the image to be detected by the residual neural network in the semantic segmentation network, the image features are input into the Deeplabv3 network to realize semantic segmentation of features corresponding to different categories of construction site soil and construction site dust in the image features. In some possible implementations, based on a preset semantic tag library, in the construction site scene area of ​​the image to be detected, based on the preset semantic tag library, the preset semantic tags matching each category of construction site soil are determined, and the preset semantic tags matching each category of construction site dust are determined.

[0073] For each construction site soil and each construction site dust, the image features are semantically segmented using matching preset semantic labels to obtain segmentation results. The preset semantic label library for construction site soil and construction site dust includes: label 0 representing the background area, label 1 representing the block construction site soil class, label 2 representing the scattered construction site soil class, label 3 representing the quasi-construction site soil stacking class, the semantic label of the dust class is 3, and the semantic label of the flying dust material stacking class is 4. If the category of the construction site soil of the image to be detected includes: block construction site soil class, and the category of construction site dust includes: flying dust material stacking class; then based on the preset semantic labels corresponding to the block construction site soil class and the flying dust material stacking class, the image to be detected is semantically segmented to obtain a segmentation result that divides the image to be detected into two areas including the block construction site soil class, the flying dust material stacking class and the background.

[0074] Step S152: determining the type of the construction site soil and the type of the construction site dust based on the segmentation result.

[0075] In some embodiments, the categories of construction site soil include at least: loose construction site soil and block construction site soil. Among them, loose construction site soil includes: loose materials such as sand, soil, gravel, planting soil, loam, clay, dry loess and clay containing a small amount of gravel; block construction site soil includes: silt, dry silt, clay containing a large amount of gravel, clay blocks, hard clay, gravelly clay and pebble-containing clay and other block materials with sudden hard structures. The categories of construction site dust include: dust category and flying dust material stacking category. Since the segmentation result is obtained based on the preset semantic labels representing the construction site soil category and the construction site dust category, the segmentation result has a preset semantic label corresponding to each category, so that the category of the construction site soil and the category of the construction site dust in the construction site scene area can be determined based on the label. For example, the segmentation results include multiple categories of construction site soil, multiple categories of construction site dust and segmentation results of other areas of the image; then through the matching relationship between the preset semantic labels and the categories of construction site soil and construction site dust, the probability value corresponding to each category of construction site soil and the probability value corresponding to each category of construction site dust can be determined; thereby, the construction site soil category with the largest probability value is determined as the final category of the construction site soil in the image to be detected, and the construction site dust category with the largest probability value is determined as the final category of the construction site dust in the image to be detected.

[0076] Step S153: based on the type of the construction site soil and the type of the construction site dust, at least the construction site soil in the construction site scene area is processed.

[0077] In some embodiments, by analyzing the correlation between the categories of construction site soil and the categories of construction site dust, the processing method of construction site soil and construction site dust in the construction site scene is determined, so that the processing of construction site dust and construction site soil in the construction site scene area can be achieved in a targeted manner. In this way, by using a semantic segmentation network trained based on the sample image set to segment the construction site soil and construction site dust of each category in the image to be detected, the area covered by construction site soil and construction site dust in the image can be effectively segmented, so that the categories of construction site dust and construction site soil in the construction site scene area can be more accurately determined, which is convenient for targeted processing of construction site dust and construction site soil.

[0078] In some possible implementations, by analyzing the type of construction site soil and the type of construction site dust, it is determined whether the dust comes from the construction site soil, and then a matching alarm message is generated. That is, the above step S153 can be implemented by the following steps:

[0079] The first step is to determine the correlation between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust.

[0080] According to the category of construction site soil and the category of construction site dust, determine whether the construction site soil can produce construction site dust of that category; if the construction site soil can produce construction site dust of that category, it means that the construction site dust may be produced by the construction site soil, and it is determined that there is a correlation between the construction site soil and construction site dust.

[0081] In some possible implementations, first, based on the type of the construction site soil, a target type of dust generated by the type of construction site soil is determined. For example, if the type of the construction site soil is sand, the type of construction site dust that can be generated is flying dust; if the type of the construction site soil is silt, no flying dust will be generated.

[0082] Then, the association relationship is determined based on the target category and the category of the construction site dust.

[0083] By determining the matching degree between the target category and the category of construction site dust, it is determined whether the construction site dust is related to the construction site soil. In this way, by analyzing the category of construction site dust that can be generated by construction site soil, it is possible to accurately determine whether the construction site dust comes from the construction site soil, which makes it easier to determine a more appropriate treatment method for the construction site scene area.

[0084] In some possible implementations, when the target category includes the category of the construction site dust, it is determined that the construction site dust is related to the construction site soil; for example, the target category is the same as the category of the construction site dust, or the target category includes two categories, dust and flying dust, and the category of the construction site dust is flying dust, then it is determined that the construction site dust may be generated by the construction site soil, that is, it is determined that the construction site dust is related to the construction site soil.

[0085] Alternatively, when the target category does not include the category of the construction site dust, it is determined that the construction site dust is not related to the construction site soil. For example, by analyzing the category of the construction site soil, it is determined that the construction site soil of this category does not generate dust or flying dust, then the category of dust that can be generated by the construction site soil obviously does not include the category of construction site dust, and it is determined that the construction site dust does not come from the construction site soil, that is, it is determined that the construction site dust is not related to the construction site soil. In this way, by analyzing the similarity between the target category of dust generated by the construction site soil and the category of construction site dust in the image, it is possible to more clearly and accurately determine whether the construction site dust and soil are related.

[0086] In the second step, when the association relationship indicates that the construction site dust is related to the construction site soil, a first alarm message is generated based on the type of the construction site soil to prompt the construction site soil to be processed.

[0087] If the target category of dust generated by construction site soil includes the category of construction site dust, it means that the construction site dust may be generated by construction site soil. Based on this, according to the category of the construction site soil, a first alarm message carrying the category and the location information of the construction site soil is generated. The alarm message can be output to the user terminal device to prompt the management personnel of the construction site scene area to deal with this category of construction site soil; or, the first alarm message is fed back to the equipment (such as a forklift) that processes the construction site scene area to control the equipment to automatically process this category of construction site soil. In this way, when there is a correlation between construction site soil and construction site dust, the alarm information of the construction site soil category is uploaded to the construction site management system so that the management personnel can deal with the construction site soil area in a timely manner.

[0088] In some embodiments, different alarm information is generated according to different associations between construction site soil and construction site dust, so as to adopt different management methods to handle construction site soil and / or construction site dust, including the following situations:

[0089] First, in the case of construction site soil and construction site dust, the management personnel can be prompted to handle the construction site soil according to the type of construction site soil, which can be achieved through the following steps:

[0090] In the first step, when the association relationship indicates that the construction site dust is related to the construction site soil, first position information of the construction site soil is determined based on the segmentation result.

[0091] In some embodiments, the position information of each category of construction site soil can be understood as the coordinate value of each category of construction site soil in the world coordinate system. By analyzing the segmentation results, it is determined which categories of construction site soil are included in the image to be detected, and through position regression, the two-dimensional coordinate value of each category of construction site soil in the image is determined, and the two-dimensional coordinate value is converted into a three-dimensional coordinate in the camera coordinate system. Through the correspondence between the camera coordinate system and the world coordinate system, the three-dimensional coordinate value of the category of construction site soil in the world coordinate system is determined. For example, by analyzing the segmentation results, it is determined that the construction site soil covered in the image to be detected is: scattered construction site soil type, then the area covered with scattered construction site soil in the segmentation results is subjected to position regression to determine the position of the area covered with scattered construction site soil in the image, and the coordinate value of the position in the world coordinate system.

[0092] In the second step, based on the first location information and the category of the construction site soil, the first alarm information is generated to prompt the construction site soil to be processed based on the first location information in a manner that matches the category of the construction site soil.

[0093] In some embodiments, the first location information and the category of the construction site soil are carried in the alarm information and fed back to the management system of the construction site scene, prompting the management personnel to process the construction site soil at the first location information in a manner that matches the category of the construction site soil; in this way, based on the location and category of the construction site soil, alarm information matching the category is generated to facilitate the management personnel to manage the construction site soil covered by the location at that location.

[0094] In some possible implementations, when the construction site soil is silt, the location of the area covered with silt is determined by performing position regression on the segmentation results, and an alarm message is generated to remind the management personnel that the location is covered with silt; or, the alarm message is fed back to the control device so that the control device can control the location covered with silt. In another implementation, the location information covered by the construction site soil is reported to the construction site soil cover warning system so that the construction site soil cover warning system generates and outputs an alarm message, thereby prompting the management personnel to deal with the area covered with the preset category of construction site soil in a timely manner.

[0095] In an embodiment of the present application, the location information of each type of construction site soil is determined by analyzing the segmentation results, and alarm information is generated and output for the construction site soil, so that the construction site soil of this type can be managed and controlled in a timely and effective manner.

[0096] Second, when the construction site soil and construction site dust are unrelated, the management personnel can be prompted to handle the construction site soil and construction site dust according to the types of construction site soil and construction site dust. This can be achieved through the following steps:

[0097] In the first step, when the association relationship indicates that the construction site dust is unrelated to the construction site soil, based on the segmentation result, first position information of the construction site soil and second position information of the construction site dust are determined.

[0098] Here, the first position information of the construction site soil and the second position information of the construction site dust may be determined by adopting the method of determining the first position information of the construction site soil based on the segmentation result in the case of the construction site soil and the construction site dust.

[0099] The second step is to generate second alarm information based on the first position information, the type of the construction site soil, the second position information and the type of the construction site dust, so as to prompt the construction site soil and the construction site dust to be processed respectively.

[0100] Since construction site soil and construction site dust are unrelated, managers are reminded to deal with construction site soil and construction site dust separately, which can more fully complete the management of the construction site scene area.

[0101] In some possible implementations, the first location information and the category of the construction site soil are taken as a pair, and the second location information and the category of the construction site dust are taken as a pair, and are carried in the second alarm information; the second alarm information is fed back to the management system of the construction site scene, prompting the management personnel to process the construction site soil at the first location information and the construction site dust at the second location information respectively; in this way, based on the location and category of the construction site soil, as well as the location and category of the construction site dust, the second alarm information is generated, so that the management personnel can manage the construction site soil and construction site dust more accurately.

[0102] In some embodiments, based on the detected construction site dust or construction site soil, an alarm message is generated, including the following two situations:

[0103] First, if construction site dust is not detected in the image to be detected, but construction site soil is detected, an alarm is generated according to the location and type of the construction site soil. The process is as follows:

[0104] The first step is to detect the construction site soil in the construction site scene area in the image to be detected when the construction site dust is not detected.

[0105] If the construction site dust is not detected in the image to be detected, the construction site soil in the construction site scene area continues to be detected.

[0106] In the second step, in response to detecting the construction site soil in the image to be detected, first warning information is generated based on first position information of the construction site soil in the image to be detected and the category of the construction site soil.

[0107] If construction site dust is not detected in the image to be detected but construction site soil is detected, the location and type of the construction site soil are carried in the first alarm information and fed back to the management system of the construction site scene area.

[0108] Second, if construction site dust is detected but construction site soil is not detected, an alarm is generated according to the location and type of construction site dust. The process is as follows:

[0109] In response to detecting the construction site dust and not detecting the construction site soil, a third alarm message is generated based on the second location information of the construction site dust and the category of the construction site dust, to prompt the construction site dust to be processed based on the second location information in a manner matching the category of the construction site dust.

[0110] Here, the third warning information carries the location and type of the construction site dust, so as to prompt the management personnel of the construction site scene area to deal with the construction site dust at the location.

[0111] In the embodiment of the present application, different alarm information is generated based on different detected contents, so that the actual situation of the construction site scene area can be fed back to the management personnel in a timely manner.

[0112] Below, an exemplary application of an embodiment of the present application in an actual application scenario will be described, taking the segmentation of construction site soil and construction site dust in an urban scene by using the created label library as an example.

[0113] In related technologies, construction site soil covering and construction site dust warning mainly rely on manpower, which often leads to high enforcement costs and low efficiency. The new intelligent construction site soil covering and construction site dust warning system can greatly reduce labor costs and achieve efficient management around the clock. In addition, the accurate segmentation and positioning of various areas in urban scenes is the basis of the construction site soil covering and construction site dust warning system.

[0114] However, in actual applications, due to the lack of in-depth discussion and analysis of construction site soil cover and construction site dust problems, it is impossible to unify the model definition and form a practical and usable labeling system.

[0115] Based on this, an embodiment of the present application provides a construction site scene detection method, which creates a unified and standardized semantic tag library for construction site soil and construction site dust in urban scenes. The semantic tag library can comprehensively cover the construction site soil coverage and construction site dust scenes, meet actual needs, and the semantic tag library system is independent, complete and basic. Based on the above semantic tag library, data definition and annotation specifications are constructed to obtain a lightweight semantic segmentation network, and the regional location of the construction site soil and construction site dust in the scene is determined according to the segmentation results. The implementation process of the construction site scene detection method is as follows:

[0116] The first step is to define the semantic labels and corresponding output results of different types of construction site soil and construction site dust according to actual needs (corresponding to the needs to be processed in the above embodiment), create semantic labels for construction site soil and construction site dust, and obtain a semantic label library.

[0117] In some embodiments, in the process of labeling different categories of construction site soil and construction site dust, since the area of ​​the construction site soil and construction site dust coverage area is irregular in shape and may be an area of ​​any shape, the contour of the area is used to label the type of construction site soil and construction site dust; when the construction site soil in the image is blocked by other objects, the various types of construction site soil and construction site dust in the image can be labeled by imaginary labeling. In order to increase the labeling speed, polygons are used in the embodiment of the present application to label the types of construction site soil and construction site dust. Among them, the semantic labels and corresponding output results of the labeled types of construction site soil and construction site dust include:

[0118] a) The category ID is 0, the semantic label is the background area, and the corresponding output result is the area outside the construction site soil and dust in the image.

[0119] b) The category id is 1, the semantic label is the area where lumps of construction site soil are piled up, and the corresponding output result is the area where lumps of construction site soil are piled up.

[0120] c) The category id is 2, the semantic label is the scattered construction site soil stacking area, and the corresponding output result is the area where the scattered construction site soil is stacked.

[0121] e) The category id is 3, the semantic label is construction site dust area, and the corresponding output result is the area where dust occurs, etc.

[0122] f) The category id is 4, the semantic label is the flying dust material stacking area, and the corresponding output result is that flying dust materials are stacked in this area.

[0123] The second step is to perform semantic segmentation on the input image to be detected based on the semantic tag library to obtain a semantic segmentation map.

[0124] In some possible implementations, the format of the output result can be a two-dimensional matrix of the same size as the original image, which is used to represent the semantic segmentation map. The semantic segmentation map includes IDs representing different construction site soil types and different construction site dust types, such as five numbers 0, 1, 2, 3 and 4, and colors representing each construction site soil type and different construction site dust types, respectively representing the background, block construction site soil stacking area, scattered construction site soil stacking area, construction site dust area and flying dust material stacking area.

[0125] The third step is to determine the location of each type of construction site soil and each type of construction site dust in the image based on the semantic segmentation map.

[0126] The construction site scene detection method provided in the embodiment of the present application can be Figure 3 The steps shown are implemented, Figure 3 Another implementation flow diagram of the construction site scene detection method provided in the embodiment of the present application, combined with Figure 3 The steps shown are explained below:

[0127] Step S301, obtaining an image to be detected, the image content of which includes construction site soil and construction site dust.

[0128] Step S302: input the image into a residual neural network for feature extraction.

[0129] Step S303: input the extracted features into a convolutional network model to perform semantic segmentation to obtain a semantic segmentation result.

[0130] In some possible implementations, the convolutional network model may be a spatial pyramid structure model with atrous convolution (Atrous Spatial Pyramid Pooling, ASPP), which may be implemented using a Deeplab v3 network.

[0131] Step S304, upsampling the semantic segmentation result to obtain an upsampling result.

[0132] Step S305: output the location area of ​​each type of construction site soil and construction site dust based on the upsampling result.

[0133] In some possible implementations, after the semantic segmentation result is upsampled, that is, the semantic segmentation result is amplified, so that the final detection result corresponding to the semantic segmentation result can be obtained based on the matching relationship between the semantic label and the output result. Figure 4 As shown, Figure 4 A schematic diagram of an application scenario of the construction site scene detection method provided in an embodiment of the present application, wherein image 401 is a collected image to be detected, and in the image 401, it includes: a dust area 411 and a block soil area 412, and the image 401 is input into the residual neural network for feature extraction, and then the feature extraction result is input into the Deeplab v3 network to perform semantic segmentation on the dust area 411 and the block soil area 412 in the image 401 to obtain a semantic segmentation result; the semantic segmentation result includes dust areas, block soil areas, and background areas distinguished by different colors.

[0134] like Figure 5 As shown, Figure 5 A schematic diagram of another application scenario of the construction site scene detection method provided in an embodiment of the present application, wherein image 501 is a collected image to be detected, and in the image 501, it includes: a dust area 511 and a loose soil area 512, and the image 501 is input into the residual neural network for feature extraction, and then the feature extraction result is input into the Deeplab v3 network to perform semantic segmentation on the dust area 511 and the loose soil area 512 in the image 501 to obtain a semantic segmentation result 513; it can be seen from the semantic segmentation result 513 that area 514 represents the position of the dust area 511 in the image 501, and area 515 represents the position of the loose soil area 512 in the image 501; the blank area 516 represents the position of the background area in the image. In this way, by standardizing the definitions of different types of construction site dust labels, the standardization and interface of sample data can be unified, so as to ensure that the created semantic label library clearly covers the construction site dust and construction site soil in the construction site scene, which is practical and easy to operate; and the positioning network used can quickly segment different types of construction site dust and construction site soil for external early warning.

[0135] In an embodiment of the present application, a complete, independent and basic semantic segmentation label system is defined by comprehensively analyzing various types of construction site dust, thereby providing a lightweight semantic segmentation algorithm framework that is easy and quick to use, while outputting standard semantic segmentation results.

[0136] In an embodiment of the present application, a complete, independent and basic semantic segmentation label system is defined by comprehensively analyzing various types of construction site soil, thereby providing a lightweight semantic segmentation algorithm framework that is easy and quick to use, while outputting standard semantic segmentation results.

[0137] The present application embodiment provides a construction site scene detection device, Figure 6 Schematic diagram of the structure of the construction site scene detection device according to the embodiment of the present application. Figure 6 As shown, the construction site scene detection device 600 includes:

[0138] A first acquisition module 601 is used to acquire an image to be detected, wherein the image to be detected includes a construction site scene area;

[0139] A first detection module 602 is used to detect construction site dust in the construction site scene area in the image to be detected;

[0140] The second detection module 603 is used to detect the construction site soil in the construction site scene area in at least one of the following frames of images in response to detecting the construction site dust: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected.

[0141] In some embodiments, the second detection module 603 includes:

[0142] A first detection submodule, configured to detect construction site soil in the construction site scene area in the image to be detected in response to detecting the construction site dust;

[0143] A first determination submodule, configured to determine at least one frame of image temporally adjacent to the image to be detected in response to the construction site soil not being detected in the image to be detected;

[0144] The second detection submodule is used to detect the construction site soil in the construction site scene area in the at least one adjacent frame image.

[0145] In some embodiments, the first detection module 602 includes:

[0146] A first extraction submodule is used to extract features from the image to be detected to obtain image features;

[0147] A third detection submodule is used to detect target image features matching preset dust features in the image features;

[0148] The second determining submodule is configured to determine, in response to detecting the target image feature, that the construction site dust is detected in the construction site scene area.

[0149] In some embodiments, the apparatus further comprises:

[0150] A first segmentation module is used for, in response to detecting the construction site soil, using a semantic segmentation network to perform scene segmentation on the construction site soil and the construction site dust in the construction site scene area to obtain a segmentation result; wherein the semantic segmentation network is obtained by training a sample image set in which different categories of construction site dust and different categories of construction site soil are annotated in the picture;

[0151] A second determination module, configured to determine the type of the construction site soil and the type of the construction site dust based on the segmentation result;

[0152] The first processing module is used to process at least the construction site soil in the construction site scene area based on the category of the construction site soil and the category of the construction site dust.

[0153] In some embodiments, the first processing module includes:

[0154] A third determination submodule, configured to determine the association relationship between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust;

[0155] The first generating submodule is used to generate first alarm information based on the type of the construction site soil when the association relationship indicates that the construction site dust is related to the construction site soil, so as to prompt the construction site soil to be processed.

[0156] In some embodiments, the third determining submodule includes:

[0157] A first determining unit is used to determine a target category of dust generated by the category of the construction site soil based on the category of the construction site soil;

[0158] The second determining unit is used to determine the association relationship based on the target category and the category of the construction site dust.

[0159] In some embodiments, the second determining unit includes:

[0160] A first determination subunit is used to determine that the construction site dust is related to the construction site soil when the target category includes the category of the construction site dust;

[0161] The second determining subunit is configured to determine that the construction site dust is not related to the construction site soil when the target category does not include the category of the construction site dust.

[0162] In some embodiments, the first generation submodule includes:

[0163] A third determining unit is configured to determine first position information of the construction site soil based on the segmentation result when the association relationship indicates that the construction site dust is related to the construction site soil;

[0164] The first generating unit is used to generate the first warning information based on the first location information and the category of the construction site soil, so as to prompt the construction site soil to be processed based on the first location information in a manner matching the category of the construction site soil.

[0165] In some embodiments, the apparatus further comprises:

[0166] A third determination module is used to determine the first position information of the construction site soil and the second position information of the construction site dust based on the segmentation result when the association relationship indicates that the construction site dust and the construction site soil are not related;

[0167] The first generating module is used to generate second alarm information based on the first position information, the type of the construction site soil, the second position information and the type of the construction site dust, so as to prompt the construction site soil and the construction site dust to be processed respectively.

[0168] In some embodiments, the apparatus further comprises:

[0169] A third detection module is used to detect the construction site soil in the construction site scene area in the image to be detected when the construction site dust is not detected;

[0170] The second generating module is used to generate first warning information based on first position information of the construction site soil in the image to be detected and the category of the construction site soil in response to detecting the construction site soil in the image to be detected.

[0171] In some embodiments, the apparatus further comprises:

[0172] The third generating module is used to generate a third alarm message in response to detecting the construction site dust and not detecting the construction site soil, based on the second position information of the construction site dust and the category of the construction site dust, so as to prompt the construction site dust to be processed based on the second position information in a manner matching the category of the construction site dust.

[0173] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0174] It should be noted that in the embodiments of the present application, if the above-mentioned construction site scene detection method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a sports hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0175] Correspondingly, an embodiment of the present application further provides a computer program product, which includes computer executable instructions. After the computer executable instructions are executed, the construction site scene detection method provided in the embodiment of the present application can be implemented.

[0176] An embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored. When the computer executable instructions are executed by a processor, the construction site scene detection method provided in the above embodiment is implemented.

[0177] The present application embodiment provides a computer device, Figure 7 Schematic diagram of the composition structure of the computer device of the embodiment of the present application, such as Figure 7 As shown, the computer device 700 includes: a processor 701, at least one communication bus, a communication interface 702, at least one external communication interface and a memory 703. The communication interface 702 is configured to realize the connection and communication between these components. The communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface. The processor 701 is configured to execute the image processing program in the memory to implement the steps of the construction site scene detection method provided in the above embodiment.

[0178] The description of the above construction site scene detection device, computer equipment and storage medium embodiments is similar to the description of the above method embodiments, and has similar technical descriptions and beneficial effects as the corresponding method embodiments. Due to space limitations, the above method embodiments can be used as examples, so they will not be repeated here. For technical details not disclosed in the construction site scene detection device, computer equipment and storage medium embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0179] It should be understood that the "one embodiment" or "one embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence number of the embodiment of the present application is only for description and does not represent the advantages and disadvantages of the embodiment. It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the element.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0181] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0182] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the above storage medium includes: mobile storage devices, read-only memory (ROM), disks or optical disks, etc. Various media that can store program codes.

[0183] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art can be stored in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks. The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, and they should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A construction site scene detection method, characterized in that: The method comprises: Acquire an image to be detected, wherein the image to be detected includes a construction site scene area; In the image to be detected, detecting construction site dust in the construction site scene area; In response to detecting the construction site dust, detecting construction site soil in the construction site scene area in at least one of the following frames: the image to be detected, and at least one frame of image that is temporally adjacent to the image to be detected; Wherein, in response to detecting the construction site dust, the construction site soil in the construction site scene area is detected in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected, including: In response to detecting the construction site dust, detecting construction site soil in the construction site scene area in the image to be detected; In response to the construction site soil not being detected in the image to be detected, determining at least one frame of image that is temporally adjacent to the image to be detected; In the at least one adjacent frame of images, construction site soil in the construction site scene area is detected.

2. The method according to claim 1, characterized in that: The detecting of construction site dust in the construction site scene area in the image to be detected includes: Extracting features from the image to be detected to obtain image features; Among the image features, detecting target image features that match preset dust features; In response to detecting the target image feature, it is determined that the construction site dust is detected in the construction site scene area.

3. The method according to claim 1 or 2, characterized in that: In response to detecting the construction site dust, detecting the construction site soil in the construction site scene area in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected, the method further includes: In response to detecting the construction site soil, a semantic segmentation network is used to perform scene segmentation on the construction site soil and the construction site dust in the construction site scene area to obtain a segmentation result; wherein the semantic segmentation network is obtained by training a sample image set in which different categories of construction site dust and different categories of construction site soil are annotated in the picture; Based on the segmentation result, determining the category of the construction site soil and the category of the construction site dust; Based on the category of the construction site soil and the category of the construction site dust, at least the construction site soil in the construction site scene area is processed.

4. The method according to claim 3, characterized in that The processing of at least the construction site soil in the construction site scene area based on the type of the construction site soil and the type of the construction site dust includes: Based on the category of the construction site soil and the category of the construction site dust, determining a correlation relationship between the construction site soil and the construction site dust; In the case where the association relationship indicates that the construction site dust is related to the construction site soil, a first alarm message is generated based on the type of the construction site soil to prompt processing of the construction site soil.

5. The method according to claim 4, characterized in that The determining the correlation between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust includes: Based on the category of the construction site soil, determining a target category of dust generated by the category of the construction site soil; The association relationship is determined based on the target category and the category of the construction site dust.

6. The method according to claim 5, characterized in that The determining the association relationship based on the target category and the category of the construction site dust includes: In a case where the target category includes the category of the construction site dust, determining that the construction site dust is related to the construction site soil; When the target category does not include the category of the construction site dust, it is determined that the construction site dust is not related to the construction site soil.

7. The method according to claim 4, characterized in that In the case where the association relationship indicates that the construction site dust is related to the construction site soil, generating first alarm information based on the type of the construction site soil to prompt processing of the construction site soil includes: In a case where the association relationship indicates that the construction site dust is related to the construction site soil, determining first position information of the construction site soil based on the segmentation result; Based on the first location information and the category of the construction site soil, the first alarm information is generated to prompt the construction site soil to be processed based on the first location information in a manner that matches the category of the construction site soil.

8. The method according to claim 4, characterized in that After determining the association relationship between the construction site soil and the construction site dust based on the category of the construction site soil and the category of the construction site dust, the method further includes: In a case where the association relationship indicates that the construction site dust is not related to the construction site soil, determining first position information of the construction site soil and second position information of the construction site dust based on the segmentation result; Based on the first position information, the type of the construction site soil, the second position information and the type of the construction site dust, second alarm information is generated to prompt the user to process the construction site soil and the construction site dust respectively.

9. The method according to claim 1 or 2, characterized in that: After detecting the construction site dust in the construction site scene area in the image to be detected, the method further includes: In the case where the construction site dust is not detected, detecting the construction site soil in the construction site scene area in the image to be detected; In response to detecting the construction site soil in the image to be detected, first warning information is generated based on first position information of the construction site soil in the image to be detected and the category of the construction site soil.

10. The method according to claim 1 or 2, characterized in that: In response to detecting the construction site dust, detecting the construction site soil in the construction site scene area in at least one of the following frames: the image to be detected, and at least one frame of image temporally adjacent to the image to be detected, the method further includes: In response to detecting the construction site dust and not detecting the construction site soil, a third alarm message is generated based on the second location information of the construction site dust and the category of the construction site dust, to prompt the construction site dust to be processed based on the second location information in a manner matching the category of the construction site dust.

11. A construction site scene detection device, characterized in that: The device comprises: A first acquisition module, used to acquire an image to be detected, wherein the image to be detected includes a construction site scene area; A first detection module, used to detect construction site dust in the construction site scene area in the image to be detected; The second detection module is used to detect the construction site soil in the construction site scene area in the image to be detected in response to the detection of the construction site dust; in response to the failure to detect the construction site soil in the image to be detected, determine at least one frame of image that is temporally adjacent to the image to be detected; and detect the construction site soil in the construction site scene area in the at least one adjacent frame of image.

12. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which, after being executed, can implement the construction site scene detection method described in any one of claims 1 to 10.

13. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores computer executable instructions, and the processor can implement the construction site scene detection method described in any one of claims 1 to 10 when running the computer executable instructions on the memory.

Images