A foundation pit-based task pushing method and device and a storage medium

Abstract

The application provides a foundation pit-based task pushing method and device and a storage medium. The method comprises the following steps: collecting multiple frames of image data in a construction site; performing semantic segmentation on the image data to obtain a foundation pit and multiple workers; counting the distance between each worker and the edge of the foundation pit in the image data; grouping the coordinates of the same worker in the multiple frames of image data into a first work sequence and grouping multiple distances into a second work sequence; grouping multiple first work sequences into a first work matrix and grouping multiple second work sequences into a second work matrix for the multiple workers; loading a work classification network; inputting the first work matrix and the second work matrix into the work classification network to identify the work mode of the multiple workers; and constructing an inspection task according to the work mode and pushing the inspection task to a manager for execution. The embodiment can push the inspection task to the manager in real time, perform inspection on the foundation pit site, and ensure the safety of construction work.

Description

TECHNICAL FIELD

[0001] The embodiment of the application belongs to the technical field of natural language processing, and particularly relates to a task pushing method based on a foundation pit, a device and a storage medium. BACKGROUND

[0002] With the continuous improvement of economic technology, the quality of the entire engineering construction is also higher, among which, the foundation pit faces certain risks such as collapse, burial, falling from a high place, mechanical injury and electric shock, and the safety management work is also paid more and more attention.

[0003] Management personnel are usually arranged on the construction site to browse the monitoring video of the construction site in real time, and when it is found that the workers tend to work near the foundation pit, the workers are warned on site.

[0004] This mode has high labor cost, and the management personnel are tired after a long time of browsing the monitoring video, and mistakes are prone to occur, thereby increasing the risk brought by the foundation pit. SUMMARY

[0005] Therefore, the embodiment of the application provides a task pushing method based on a foundation pit, a device and a storage medium, so as to reduce the risk brought by the foundation pit.

[0006] The first aspect of the embodiment of the application provides a task pushing method based on a foundation pit, comprising:

[0007] Collecting multiple image data in a construction site;

[0008] Performing semantic segmentation on the image data to obtain a foundation pit and multiple workers;

[0009] Counting the distance between each worker and the edge of the foundation pit in the image data;

[0010] Grouping the coordinates of the same worker in multiple image data into a first work sequence, and grouping multiple distances into a second work sequence;

[0011] For multiple workers, grouping multiple first work sequences into a first work matrix, and grouping multiple second work sequences into a second work matrix;

[0012] Loading a work classification network;

[0013] Inputting the first work matrix and the second work matrix into the work classification network to identify the work mode of multiple workers;

[0014] Constructing an inspection task according to the work mode, and pushing the inspection task to a management personnel for execution.

[0015] Optionally, the job classification network comprises a first encoder, a second encoder, a third encoder, a fourth encoder, a feature interaction module, a feature fusion module and a head structure.

[0016] The inputting the first job matrix and the second job matrix into the job classification network to identify job modes of the plurality of job workers comprises:

[0017] The first job matrix is inputted into the first encoder to encode and obtain first job features.

[0018] The second job matrix is inputted into the second encoder to encode and obtain second job features.

[0019] The first job features and the second job features are inputted into the feature interaction module to interact partial features and obtain third job features and fourth job features.

[0020] The third job features are inputted into the third encoder to encode and obtain fifth job features.

[0021] The fourth job features are inputted into the fourth encoder to encode and obtain sixth job features.

[0022] The fifth job features and the sixth job features are inputted into the feature fusion module to fuse into seventh job features.

[0023] The seventh job features are inputted into the head structure to classify and obtain the job modes of the plurality of job workers.

[0024] Optionally, the first encoder comprises three first convolutional layers, and the second encoder comprises three second convolutional layers.

[0025] The inputting the first job matrix into the first encoder to encode and obtain first job features comprises:

[0026] The first job matrix is sequentially inputted into three first convolutional layers to perform convolutional operations and obtain first job features.

[0027] The inputting the second job matrix into the second encoder to encode and obtain second job features comprises:

[0028] The second job matrix is sequentially inputted into three second convolutional layers to perform convolutional operations and obtain second job features.

[0029] Optionally, the inputting the first job features and the second job features into the feature interaction module to interact partial features and obtain third job features and fourth job features comprises:

[0030] performing an up-sampling operation on the first job feature to obtain a first intermediate feature;

[0031] adding the first intermediate feature and the second job feature to obtain a third job feature;

[0032] performing a down-sampling operation on the third job feature to obtain a second intermediate feature;

[0033] adding the second intermediate feature and the first intermediate feature to obtain a fourth job feature.

[0034] Optionally, the third encoder comprises three third convolutional layers, and the fourth encoder comprises three fourth convolutional layers;

[0035] the encoding of the third job feature into the third encoder to obtain a fifth job feature comprises:

[0036] sequentially inputting the third job feature into the three third convolutional layers to perform convolutional operations to obtain the fifth job feature;

[0037] the encoding of the fourth job feature into the fourth encoder to obtain a sixth job feature comprises:

[0038] sequentially inputting the fourth job feature into the three fourth convolutional layers to perform convolutional operations to obtain the sixth job feature.

[0039] Optionally, the feature fusion module comprises a fifth convolutional layer and a sixth convolutional layer;

[0040] the fusion of the fifth job feature and the sixth job feature into the feature fusion module to obtain a seventh job feature comprises:

[0041] inputting the fifth job feature into the fifth convolutional layer to perform convolutional layer operations to obtain a third intermediate feature;

[0042] splicing the third intermediate feature and the sixth job feature to obtain a fourth intermediate feature;

[0043] inputting the fourth intermediate feature into the sixth convolutional layer to perform convolutional layer operations to obtain the seventh job feature.

[0044] Optionally, the constructing of the inspection task according to the job mode comprises:

[0045] generating job description information using the first job matrix and the second job matrix;

[0046] combining the job mode and the job description information into target job information;

[0047] searching a task record information similar to the target operation information in a preset task library; the task record information is information recorded when the inspection task is historically executed;

[0048] constructing an inspection task according to the task record information and the target operation information.

[0049] Optionally, the constructing an inspection task according to the task record information and the target operation information comprises:

[0050] extracting a task point in the task record information;

[0051] writing the task point and the target operation information into a preset template to obtain a guide language;

[0052] inputting the guide language into a large language model to construct an inspection task.

[0053] A second aspect of the embodiment of the application provides a task pushing device based on a foundation pit, comprising:

[0054] an image data acquisition module, configured to acquire multiple frames of image data in a construction site;

[0055] a semantic segmentation module, configured to perform semantic segmentation on the image data to obtain a foundation pit and multiple operation personnel;

[0056] a distance detection module, configured to count distances between each of the operation personnel and edges of the foundation pit in the image data;

[0057] a sequence composition module, configured to compose coordinates of a same operation personnel in multiple frames of the image data into a first operation sequence and compose multiple distances into a second operation sequence;

[0058] a matrix composition module, configured to compose multiple first operation sequences into a first operation matrix and compose multiple second operation sequences into a second operation matrix for multiple operation personnel;

[0059] an operation classification network loading module, configured to load an operation classification network;

[0060] an operation mode recognition module, configured to input the first operation matrix and the second operation matrix into the operation classification network to recognize operation modes of the multiple operation personnel;

[0061] an inspection task processing module, configured to construct an inspection task according to the operation modes and push the inspection task to a management personnel for execution.

[0062] Optionally, the operation classification network comprises a first encoder, a second encoder, a third encoder, a fourth encoder, a feature interaction module, a feature fusion module and a head structure.

[0063] The job mode recognition module is further configured to:

[0064] input the first job matrix into the first encoder for coding to obtain first job features;

[0065] input the second job matrix into the second encoder for coding to obtain second job features;

[0066] input the first job features and the second job features into the feature interaction module to obtain third job features and fourth job features;

[0067] input the third job features into the third encoder for coding to obtain fifth job features;

[0068] input the fourth job features into the fourth encoder for coding to obtain sixth job features;

[0069] input the fifth job features and the sixth job features into the feature fusion module to obtain seventh job features;

[0070] input the seventh job features into the head structure for classification to obtain job modes of the plurality of job workers.

[0071] Optionally, the first encoder comprises three first convolutional layers, and the second encoder comprises three second convolutional layers.

[0072] The job mode recognition module is further configured to:

[0073] input the first job matrix into the three first convolutional layers in sequence for convolution operation to obtain first job features;

[0074] The job mode recognition module is further configured to:

[0075] input the second job matrix into the three second convolutional layers in sequence for convolution operation to obtain second job features.

[0076] Optionally, the job mode recognition module is further configured to:

[0077] perform up-sampling operation on the first job features to obtain first intermediate features;

[0078] add the first intermediate features and the second job features to obtain third job features;

[0079] perform down-sampling operation on the third job features to obtain second intermediate features;

[0080] The second intermediate feature is added to the first intermediate feature to obtain a fourth job feature.

[0081] In an embodiment of the present application, the third encoder comprises three third convolutional layers, and the fourth encoder comprises three fourth convolutional layers;

[0082] The job mode recognition module is further configured to:

[0083] The third job feature is sequentially input into the three third convolutional layers to perform convolutional operations to obtain a fifth job feature;

[0084] The fourth job feature is input into the fourth encoder to be encoded to obtain a sixth job feature, including:

[0085] The fourth job feature is sequentially input into the three fourth convolutional layers to perform convolutional operations to obtain a sixth job feature.

[0086] Optionally, the feature fusion module comprises a fifth convolutional layer and a sixth convolutional layer;

[0087] The job mode recognition module is further configured to:

[0088] The fifth job feature is input into the fifth convolutional layer to perform convolutional layer operations to obtain a third intermediate feature;

[0089] The third intermediate feature and the sixth job feature are spliced to obtain a fourth intermediate feature;

[0090] The fourth intermediate feature is input into the sixth convolutional layer to perform convolutional layer operations to obtain a seventh job feature.

[0091] Optionally, the inspection task processing module is further configured to:

[0092] Generate job description information using the first job matrix and the second job matrix;

[0093] Combine the job mode and the job description information into target job information;

[0094] Search for task record information similar to the target job information in a preset task library; the task record information is information recorded when the inspection task is historically executed;

[0095] Construct an inspection task according to the task record information and the target job information.

[0096] Optionally, the inspection task processing module is further configured to:

[0097] Extract task points from the task record information;

[0098] write the task points and the target work information into a preset template to obtain a guidance;

[0099] input the guidance into a large language model to construct an inspection task.

[0100] A third aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the task pushing method based on the foundation pit when executing the computer program.

[0101] A fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the task pushing method based on the foundation pit as described in the first aspect.

[0102] A fifth aspect of the embodiments of the present application provides a computer program product, which, when running on a computer, causes the computer to execute the task pushing method based on the foundation pit as described in the first aspect.

[0103] In the embodiment, multiple frames of image data are collected in a construction site; the image data are subjected to semantic segmentation to obtain a foundation pit and multiple workers; distances between the edges of the foundation pit and each worker are counted in the image data; coordinates of the same worker in multiple frames of image data are grouped into a first work sequence, and multiple distances are grouped into a second work sequence; for multiple workers, multiple first work sequences are grouped into a first work matrix, and multiple second work sequences are grouped into a second work matrix; a work classification network is loaded; the first work matrix and the second work matrix are input into the work classification network to identify work modes of the multiple workers; an inspection task is constructed according to the work modes, and the inspection task is pushed to a manager for execution. In the embodiment, the work modes are identified according to the movement modes of the workers relative to the foundation pit in computer vision, and the inspection task is constructed according to the work modes in natural language processing, so that the degree of automation is high, the accuracy is high, the fatigue caused by long-time browsing of monitoring videos is effectively reduced to reduce the omission, the inspection task is pushed to the manager in real time, the foundation pit site is inspected in time, the risk brought by the foundation pit is reduced, and the safety of construction work is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0104] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0105] Figure 1 is a schematic diagram of a task pushing method based on a foundation pit provided by an embodiment of the present application;

[0106] Figure 2 is a schematic diagram of a work classification network provided by an embodiment of the present application;

[0107] Figure 3 is a schematic diagram of a task pushing device based on a foundation pit provided by an embodiment of the present application;

[0108] Figure 4 is a schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0109] In the following description, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the present embodiments. However, persons skilled in the art will understand that the present application can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and methods have not been described in detail in order to avoid obscuring the present application.

[0110] The technical solutions of the present application will be described below through specific embodiments.

[0111] Referring to Figure 1 , a schematic diagram of a task pushing method based on a foundation pit provided by an embodiment of the present application is shown, which can specifically include the following steps:

[0112] Step 101, collect multiple frames of image data in a construction site.

[0113] In this embodiment, one or more cameras can be deployed on the construction site, and each camera can face the construction site to collect video data, wherein the video data has multiple frames of image data.

[0114] Step 102, perform semantic segmentation on the image data to obtain a foundation pit and multiple workers.

[0115] In this embodiment, a semantic segmentation network such as FCN (Fully Convolution Networks), U-Net, SegNet (Segmentation Networks), DeepLab (Deep Convolutional Nets), etc. can be pre-constructed and trained.

[0116] Each frame of image data is input into the semantic segmentation network for semantic segmentation to obtain a foundation pit (pixel point) and multiple workers (pixel points).

[0117] Step 103, in the image data, the distance between each worker and the edge of the foundation pit is counted.

[0118] In each frame of image data, the shortest distance (pixel distance) between each worker (pixel point) and the edge of the foundation pit (pixel point) can be counted using projection or other methods.

[0119] Step 104, the coordinates of the same worker in multiple frames of image data form a first work sequence, and the multiple distances form a second work sequence.

[0120] In this embodiment, KCF (Kernelized Correlation Filter) or other algorithms can be used to track each worker (pixel point) in multiple frames of image data.

[0121] For the same worker (pixel point), on the one hand, multiple coordinates of the worker in multiple frames of image data form a first work sequence in time order, and on the other hand, multiple distances of the worker in multiple frames of image data form a second work sequence in time order.

[0122] The first work sequence and the second work sequence reflect the movement rule of a single worker during work, and can reflect the characteristics of the work mode to a certain extent.

[0123] Step 105, for multiple workers, multiple first work sequences form a first work matrix, and multiple second work sequences form a second work matrix.

[0124] In this embodiment, the multiple workers can be configured in order, such as from left to right, from top to bottom, etc., and the multiple first work sequences form a first work matrix according to the order of the workers, and the multiple second work sequences form a second work matrix according to the order of the workers.

[0125] The first work sequence and the second work sequence are both sequence data of a single worker. Considering that the construction site is basically a collaborative work of multiple workers, the multiple first work sequences form a first work matrix, and the multiple second work sequences form a second work matrix, which can improve the collaborative features of multiple workers in the first work matrix and the second work matrix, and improve the accuracy of identifying the work mode.

[0126] Step 106, load the work classification network.

[0127] In the embodiment, the job classification network can be constructed and trained in advance based on deep learning. The job classification network is a multi-classification neural network. The structure of the job classification network is not limited to a manually designed neural network, but can also be a neural network optimized by a model quantization method, a neural network searched for the characteristics of building construction by a NAS (Neural Architecture Search) method, and the like. The embodiment is not limited thereto.

[0128] In training the job classification network, the historical first job matrix and the second job matrix are selected as samples, the job mode labeled by the manager is taken as a label Label, a cross-entropy is selected as a loss function, and the job classification network is supervised and trained, so that the job classification network has the ability to classify the job mode of the job personnel.

[0129] The job mode is set for the foundation pit, for example, staying near the foundation pit, construction near the foundation pit, staying away from the foundation pit, construction away from the foundation pit, and the like.

[0130] Step 107, inputting the first job matrix and the second job matrix into the job classification network to identify the job mode of the plurality of job personnel.

[0131] In actual application, the first job matrix and the second job matrix are scaled to a specified size. If the scaling is completed, the first job matrix and the second job matrix are input into the job classification network. The job classification network performs a multi-classification task to identify the job mode of the plurality of job personnel when they work together.

[0132] In an embodiment of the application, as shown in Figure 2 the job classification network includes a first encoder Encoder_1, a second encoder Encoder_2, a third encoder Encoder_3, a fourth encoder Encoder_4, a feature interaction module (FIM), a feature fusion module (FFM), and a head structure Head.

[0133] Then, in the embodiment, step 107 can include the following steps:

[0134] Step 1071, inputting the first job matrix into the first encoder to encode and obtaining first job features.

[0135] In the embodiment, as shown in Figure 2 the first job matrix can be input into the first encoder Encoder_1 for preliminary encoding to obtain first job features E1.

[0136] As shown in the figure, the first encoder Encoder_1 includes three first convolutional layers Conv_1, and then in this example, the first job matrix can be sequentially input into the three first convolutional layers Conv_1 to perform convolutional operations, to obtain the first job feature E1. Figure 2

[0137] Step 1072, input the second job matrix into the second encoder to encode, to obtain the second job feature.

[0138] In this embodiment, as shown in the figure, the second job matrix can be input into the second encoder Encoder_2 to perform preliminary encoding, to obtain the second job feature E2. Figure 2

[0139] As shown in the figure, the second encoder Encoder_2 includes three second convolutional layers Conv_2, and then in this example, the second job matrix can be sequentially input into the three second convolutional layers Conv_2 to perform convolutional operations, to obtain the second job feature E2. Figure 2

[0140] Step 1073, input the first job feature and the second job feature into the feature interaction module to interact part of the features, to obtain the third job feature and the fourth job feature.

[0141] In this embodiment, as shown in the figure, the first job feature E1 and the second job feature E2 can be input into the feature interaction module FIM to interact part of the features with each other, to obtain the third job feature E3 and the fourth job feature E4. Figure 2

[0142] As shown in the figure, in the feature interaction module FIM, the first job feature E1 is subjected to an up-sampling operation UpSample to obtain a first intermediate feature, the first intermediate feature is added to the second job feature E2 by an addition operation Add to obtain the third job feature E3, the third job feature E3 is subjected to a down-sampling operation DownSample to obtain a second intermediate feature, and the second intermediate feature is added to the first intermediate feature by an addition operation Add to obtain the fourth job feature E4.

[0143] Step 1074, input the third job feature into the third encoder to encode, to obtain the fifth job feature.

[0144] In this embodiment, as shown in the figure, the third job feature E3 can be input into the third encoder Encoder_3 to perform deep-level encoding, to obtain the fifth job feature E5. Figure 2

[0145] As shown in the figure, the third encoder Encoder_3 includes three third convolutional layers Conv_3, and then in this example, the third job feature E3 can be sequentially input into the three third convolutional layers Conv_3 to perform convolutional operations, to obtain the fifth job feature E5. Figure 2 ​​​​​As shown, the third encoder Encoder_3 includes three third convolutional layers Conv_3. In the example, the third job feature E3 is sequentially input into the three third convolutional layers Conv_3 to perform convolution operations, resulting in the fifth job feature E5.

[0146] Step 1075: Input the fourth job feature into the fourth encoder for encoding to obtain the sixth job feature.

[0147] In this embodiment, as Figure 2 As shown, the fourth job feature can be input into the fourth encoder Encoder_4 for deep encoding to obtain the sixth job feature.

[0148] For example, such as Figure 2 As shown, the fourth encoder Encoder_4 includes three fourth convolutional layers Conv_4. In this example, the fourth job feature E4 is sequentially input into the three fourth convolutional layers Conv_4 to perform convolution operations, resulting in the sixth job feature E6.

[0149] Step 1076: Input the fifth and sixth task features into the feature fusion module and fuse them into the seventh task feature.

[0150] In this embodiment, as Figure 2 As shown, the fifth job feature E5 and the sixth job feature E6 are input into the feature fusion module FFM and fused into the seventh job feature E7.

[0151] For example, such as Figure 2 As shown, the feature fusion module includes a fifth convolutional layer Conv_5 and a sixth convolutional layer Conv_6. In this example, the fifth job feature E5 is input into the fifth convolutional layer Conv_5 to perform convolutional layer operations, resulting in the third intermediate feature. The third intermediate feature is concatenated with the sixth job feature E6 to form the fourth intermediate feature. The fourth intermediate feature is then input into the sixth convolutional layer Conv_6 to perform convolutional layer operations, resulting in the seventh job feature E7.

[0152] In this embodiment, the first task matrix and the second task matrix are initially encoded, then some features are interacted with each other, and then deeper encoding is carried out. Finally, they are fused together. While retaining the original coordinate features and distance features, feature fusion is carried out step by step to improve the quality of the final fused features.

[0153] Step 1077: Classify the seventh operation feature into the header structure to obtain the operation modes of multiple operators.

[0154] In this embodiment, the work patterns of multiple workers are classified in the seventh work feature E7 input header structure Head.

[0155] Exemplarily, as shown in the figure, the head structure Head includes three fully connected layers (FC), then in the embodiment, the seventh job feature E7 is sequentially input into the three fully connected layers FC to be mapped into an eighth job feature, the eighth job feature is activated using a Sigmoid or the like activation function, a plurality of probabilities that a plurality of job personnel respectively belong to a plurality of job modes are obtained, and the job mode with the highest probability is output. Figure 2

[0156] Step 108, constructing an inspection task according to the job mode, and pushing the inspection task to the management personnel for execution.

[0157] In the embodiment, the technology of NLP (Natural Language Processing) can be used to construct a suitable inspection task according to the job mode.

[0158] In a specific implementation, the first job matrix and the second job matrix can be input into a text generator such as LSTM (Long Short Term Memory), Transformer or the like, and the first job matrix and the second job matrix are used to generate job description information, and the job mode and the job description information are combined into target job information.

[0159] The target job information is encoded into a vector, and the vector is used to search for task record information similar to the target job information in a preset task library; wherein the task record information is information recorded when the historical inspection task is executed.

[0160] At this time, the inspection task can be constructed according to the task record information and the target job information.

[0161] Further, the task record information belongs to structured data, then the task points can be extracted from the task record information according to the data structure, for example, the residence time at a certain position, the warning mode to the job personnel, and the like.

[0162] The task points and the target job information are written into a preset template to obtain a guide Prompt, and the guide Prompt is input into a large language model (Large Language Model) to construct an inspection task.

[0163] ​The inspection task is pushed to the manager (account) by using short message, instant messaging message and the like, the manager performs the inspection task, inspects the foundation pit according to the requirements of the inspection task, warns the relevant workers, and when the inspection is completed, the manager can correct the work mode, obtain new samples and labels, and update the work classification network in Fine-tuning and the like mode, continuously improve the performance of the work classification network, and generate new task record information and store in the task library.

[0164] In the embodiment, multiple image data are collected in the construction site; the image data are subjected to semantic segmentation to obtain a foundation pit and multiple workers; distances between each worker and edges of the foundation pit are counted in the image data; coordinates of the same worker in the multiple image data are grouped into a first work sequence, and multiple distances are grouped into a second work sequence; for the multiple workers, multiple first work sequences are grouped into a first work matrix, and multiple second work sequences are grouped into a second work matrix; a work classification network is loaded; the first work matrix and the second work matrix are input into the work classification network to identify work modes of the multiple workers; and an inspection task is constructed according to the work modes, and the inspection task is pushed to the manager for execution. In the embodiment, the work mode is identified according to the movement mode of the worker relative to the foundation pit in computer vision, and the inspection task is constructed according to the work mode in natural language processing, so that the automation degree is high, the accuracy is high, the fatigue caused by long-time browsing of the monitoring video is effectively reduced, the inspection task is pushed to the manager in real time, the foundation pit site is timely inspected, the risk brought by the foundation pit is reduced, and the safety of the construction work is ensured.

[0165] It should be noted that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0166] Reference Figure 3 , a schematic diagram of a task pushing device based on a foundation pit provided by an embodiment of the application is shown, which can specifically include the following modules:

[0167] The image data collection module 301 is configured to collect multiple image data in the construction site.

[0168] The semantic segmentation module 302 is configured to perform semantic segmentation on the image data to obtain a foundation pit and multiple workers.

[0169] The distance detection module 303 is configured to count distances between each worker and edges of the foundation pit in the image data.

[0170] The sequence composition module 304 is configured to compose coordinates of the same work personnel in multiple frames of image data into a first work sequence and compose multiple distances into a second work sequence.

[0171] The matrix composition module 305 is configured to compose multiple first work sequences into a first work matrix and compose multiple second work sequences into a second work matrix for multiple work personnel.

[0172] The work classification network loading module 306 is configured to load a work classification network.

[0173] The work mode recognition module 307 is configured to input the first work matrix and the second work matrix into the work classification network to recognize work modes of the multiple work personnel.

[0174] The inspection task processing module 308 is configured to construct an inspection task according to the work modes and push the inspection task to a management personnel for execution.

[0175] In an embodiment of the present application, the work classification network includes a first encoder, a second encoder, a third encoder, a fourth encoder, a feature interaction module, a feature fusion module and a head structure.

[0176] The work mode recognition module 307 is further configured to:

[0177] input the first work matrix into the first encoder to encode and obtain first work features;

[0178] input the second work matrix into the second encoder to encode and obtain second work features;

[0179] input the first work features and the second work features into the feature interaction module to interact partial features and obtain third work features and fourth work features;

[0180] input the third work features into the third encoder to encode and obtain fifth work features;

[0181] input the fourth work features into the fourth encoder to encode and obtain sixth work features;

[0182] input the fifth work features and the sixth work features into the feature fusion module to fuse and obtain seventh work features;

[0183] input the seventh work features into the head structure to classify and obtain work modes of the multiple work personnel.

[0184] In an embodiment of the present application, the first encoder includes three first convolutional layers and the second encoder includes three second convolutional layers.

[0185] The job mode recognition module 307 is further configured to:

[0186] input the first job matrix into three first convolutional layers in sequence to perform convolutional operations, to obtain first job features;

[0187] The job mode recognition module 307 is further configured to:

[0188] input the second job matrix into three second convolutional layers in sequence to perform convolutional operations, to obtain second job features.

[0189] In an embodiment of the present application, the job mode recognition module 307 is further configured to:

[0190] perform up-sampling operations on the first job features to obtain first intermediate features;

[0191] add the first intermediate features and the second job features to obtain third job features;

[0192] perform down-sampling operations on the third job features to obtain second intermediate features;

[0193] add the second intermediate features and the first intermediate features to obtain fourth job features.

[0194] In an embodiment of the present application, the third encoder comprises three third convolutional layers, and the fourth encoder comprises three fourth convolutional layers;

[0195] The job mode recognition module 307 is further configured to:

[0196] input the third job features into three third convolutional layers in sequence to perform convolutional operations, to obtain fifth job features;

[0197] The inputting the fourth job features into the fourth encoder to encode, to obtain sixth job features, comprises:

[0198] input the fourth job features into three fourth convolutional layers in sequence to perform convolutional operations, to obtain sixth job features.

[0199] In an embodiment of the present application, the feature fusion module comprises a fifth convolutional layer and a sixth convolutional layer;

[0200] The job mode recognition module 307 is further configured to:

[0201] input the fifth job features into the fifth convolutional layer to perform convolutional layer operations, to obtain third intermediate features;

[0202] Splice the third intermediate feature and the sixth job feature into a fourth intermediate feature;

[0203] Input the fourth intermediate feature into the sixth convolutional layer to perform a convolutional layer operation to obtain a seventh job feature.

[0204] In an embodiment of the present application, the inspection task processing module 308 is further configured to:

[0205] Generate job description information using the first job matrix and the second job matrix;

[0206] Combine the job mode and the job description information into target job information;

[0207] Search for task record information similar to the target job information in a preset task library; the task record information is information recorded when the inspection task is historically executed;

[0208] Construct an inspection task according to the task record information and the target job information.

[0209] In an embodiment of the present application, the inspection task processing module 308 is further configured to:

[0210] Extract task points from the task record information;

[0211] Write the task points and the target job information into a preset template to obtain a guide;

[0212] Input the guide into a large language model to construct an inspection task.

[0213] The embodiment of the present application provides a task pushing device based on a foundation pit, and application of the device can realize each step in each method embodiment.

[0214] For the device embodiment, it is basically similar to the method embodiment, so it is described more simply, and the related parts refer to the description in the method embodiment part.

[0215] Referring to Figure 4 , a schematic diagram of a terminal device provided by an embodiment of the present application is shown. As shown in Figure 4 , the terminal device 400 in the embodiment of the present application includes a processor 410, a memory 420, and a computer program 421 stored in the memory 420 and executable on the processor 410. The processor 410 implements the steps in each embodiment of the above-mentioned task pushing method based on a foundation pit when executing the computer program 421. Alternatively, the processor 410 implements the functions of each module / unit in each device embodiment when executing the computer program 421.

[0216] For example, the computer program 421 can be segmented into one or more modules / units stored in the memory 420 and executed by the processor 410 to accomplish the present application. The one or more modules / units can be a series of computer program instruction segments capable of accomplishing a specific function, which can be used to describe the execution process of the computer program 421 in the terminal device 400.

[0217] The terminal device 400 can include, but is not limited to, the processor 410 and the memory 420. Those skilled in the art can understand that the terminal device 400 can include more or fewer components than those shown, or can combine some components, or include different components. For example, the terminal device 400 can also include an input / output device, a network access device, a bus, etc. Figure 4 The terminal device 400 shown in the figure is only an example and does not constitute a limitation on the terminal device 400, which can include more or fewer components than those shown, or can combine some components, or include different components. For example, the terminal device 400 can also include an input / output device, a network access device, a bus, etc.

[0218] The processor 410 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0219] The memory 420 can be an internal storage unit of the terminal device 400, such as a hard disk or a memory of the terminal device 400. The memory 420 can also be an external storage device of the terminal device 400, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 420 can include both the internal storage unit and the external storage device of the terminal device 400. The memory 420 is used to store the computer program 421 and other programs and data required by the terminal device 400. The memory 420 can also be used to temporarily store data that has been output or will be output.

[0220] The embodiment of the present application further discloses a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the task pushing method based on a foundation pit when executing the computer program.

[0221] The embodiment of the present application further discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the task pushing method based on a foundation pit.

[0222] The embodiment of the present application further discloses a computer program product, which, when running on a computer, enables the computer to execute the task pushing method based on a foundation pit.

[0223] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A task push method based on a foundation pit, characterized in that, include: Collect multiple frames of image data at the construction site; Semantic segmentation was performed on the image data to obtain the foundation pit and multiple workers; The distance between each worker and the edge of the pit is calculated from the image data. The coordinates of the same operator in multiple frames of image data are used to form a first operation sequence, and the multiple distances are used to form a second operation sequence; For multiple operators, the multiple first work sequences are combined into a first work matrix, and the multiple second work sequences are combined into a second work matrix; Load the job classification network; the job classification network includes a first encoder, a second encoder, a third encoder, a fourth encoder, a feature interaction module, a feature fusion module, and a head structure; The first job matrix is ​​input into the first encoder for encoding to obtain the first job feature; The second job matrix is ​​input into the second encoder for encoding to obtain the second job features; In the feature interaction module, an upsampling operation is performed on the first job feature to obtain a first intermediate feature; the first intermediate feature is added to the second job feature to obtain a third job feature; a downsampling operation is performed on the third job feature to obtain a second intermediate feature; and the second intermediate feature is added to the first intermediate feature to obtain a fourth job feature. The third job feature is input into the third encoder for encoding to obtain the fifth job feature; The fourth job feature is input into the fourth encoder for encoding to obtain the sixth job feature; The fifth task feature and the sixth task feature are input into the feature fusion module and fused into a seventh task feature; The seventh job feature is input into the head structure to classify and obtain the job patterns of multiple job workers; Based on the aforementioned work mode, inspection tasks are constructed and then pushed to management personnel for execution.

2. The method according to claim 1, characterized in that, The first encoder includes three first convolutional layers, and the second encoder includes three second convolutional layers; The step of inputting the first job matrix into the first encoder for encoding to obtain the first job feature includes: The first job matrix is ​​sequentially input into the three first convolutional layers to perform convolution operations, thereby obtaining the first job features; The step of inputting the second job matrix into the second encoder for encoding to obtain the second job features includes: The second job matrix is ​​sequentially input into the three second convolutional layers to perform convolution operations, thereby obtaining the second job features.

3. The method according to claim 1, characterized in that, The third encoder includes three third convolutional layers, and the fourth encoder includes three fourth convolutional layers; The step of inputting the third job feature into the third encoder to obtain the fifth job feature includes: The third job feature is sequentially input into the three third convolutional layers to perform convolution operations, resulting in the fifth job feature; The step of inputting the fourth job feature into the fourth encoder to obtain the sixth job feature includes: The fourth job feature is sequentially input into the three fourth convolutional layers to perform convolution operations, resulting in the sixth job feature.

4. The method according to claim 3, characterized in that, The feature fusion module includes a fifth convolutional layer and a sixth convolutional layer; The step of inputting the fifth job feature and the sixth job feature into the feature fusion module and fusing them into a seventh job feature includes: The fifth task feature is input into the fifth convolutional layer to perform convolutional layer operations, resulting in the third intermediate feature; The third intermediate feature is combined with the sixth task feature to form the fourth intermediate feature; The fourth intermediate feature is input into the sixth convolutional layer to perform convolutional layer operations, resulting in the seventh job feature.

5. The method according to any one of claims 1-4, characterized in that, The process of constructing inspection tasks based on the aforementioned work mode includes: Use the first job matrix and the second job matrix to generate job description information; The job mode is combined with the job description information to form target job information; Search the preset task library for task record information similar to the target task information; the task record information is information recorded when the inspection task was executed in the past. An inspection task is constructed based on the task record information and the target operation information.

6. The method according to claim 5, characterized in that, The process of constructing an inspection task based on the task record information and the target operation information includes: Extract the key points of the task from the task record information; The key points of the task and the target operation information are written into a preset template to obtain instructions; The instructions are input into a large language model to construct the inspection task.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the pit-based task push method as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the pit-based task push method as described in any one of claims 1-6.

Images