Production workshop construction operation safety early warning system and method based on the Internet of Things

Abstract

The present application relates to the field of safety warning, and specifically discloses a safety warning system and method for production workshop construction operations based on the Internet of Things. The system first obtains the surrounding environment image of the construction vehicle during its driving process collected by a camera and the ranging values ​​of the construction vehicle and the surrounding environment at multiple predetermined time points collected by a phase laser ranging sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, a classifier is used to obtain a classification result to determine whether to issue a safety warning alarm, thereby further improving the safety performance of the construction vehicle and reducing the risk of accidents.

Description

Technical Field

[0001] The present application relates to the field of safety warning, and more specifically, to a safety warning system and method for production workshop construction operations based on the Internet of Things. Background Art

[0002] Production workshop construction work refers to the process of various production process operations and equipment commissioning in the manufacturing workshop. This includes the processing of raw materials, assembly of products, maintenance and adjustment of equipment, etc. During the construction process, operators need to strictly abide by the operating procedures to ensure that the production process is carried out safely and efficiently to ensure product quality and production efficiency.

[0003] When carrying out construction and maintenance work in the production workshop, it is necessary to ensure that the construction unit does not change the location, structure, and operation mode of the existing production equipment without interrupting production. In addition, due to the uncertainty of the construction location and time, the construction tasks are random, and the engineering operations and mechanical equipment maintenance tasks are temporary. In particular, considering the characteristics of the workshop production environment, such as heavy loads, long distances, irregular and irregular back and forth operations, and high noise, unexpected situations may occur, such as cargo falling, posing a threat to construction personnel.

[0004] Therefore, a safety early warning system and method for production workshop construction operations based on the Internet of Things is desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a production workshop construction safety warning system and method based on the Internet of Things, which first obtains the surrounding environment image of the construction vehicle during the driving process collected by the camera and the ranging values ​​of the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains the classification result through the classifier to determine whether to issue a safety warning alarm, thereby further improving the safety performance of the construction vehicle and reducing the risk of accidents.

[0006] According to one aspect of the present application, a production workshop construction operation safety early warning system based on the Internet of Things is provided, which includes:

[0007] A construction vehicle driving data acquisition module is used to acquire the surrounding environment image of the construction vehicle during driving collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor;

[0008] A construction vehicle driving data extraction module is used to extract a construction vehicle surrounding environment associated feature vector and a construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle during driving collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor;

[0009] The safety warning judgment module is used to judge whether to issue a safety warning alarm based on the surrounding environment associated feature vector of the construction vehicle and the distance associated feature vector of the construction vehicle.

[0010] According to another aspect of the present application, a safety early warning method for production workshop construction operations based on the Internet of Things is provided, which includes:

[0011] Acquire the surrounding environment image of the construction vehicle during its driving process collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor;

[0012] Extracting a construction vehicle surrounding environment associated feature vector and a construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle during its driving process acquired by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points acquired by the phase laser ranging sensor;

[0013] Based on the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector, it is determined whether to issue a safety warning alarm.

[0014] Compared with the prior art, the present application provides a production workshop construction safety warning system and method based on the Internet of Things. The system first obtains the surrounding environment image of the construction vehicle during its driving process collected by a camera and the ranging values ​​of the construction vehicle and the surrounding environment at multiple predetermined time points collected by a phase laser ranging sensor. Then, deep learning technology is used to perform feature extraction and correlation analysis on the two. Finally, a classifier is used to obtain a classification result to determine whether to issue a safety warning alarm, thereby further improving the safety performance of construction vehicles and reducing accident risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1It is a block diagram of a production workshop construction operation safety early warning system based on the Internet of Things according to an embodiment of the present application.

[0017] Figure 2 It is a block diagram of a construction vehicle driving data extraction module in a production workshop construction operation safety warning system based on the Internet of Things according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a working vehicle driving environment feature extraction unit in a production workshop construction work safety warning system based on the Internet of Things according to an embodiment of the present application.

[0019] Figure 4 It is a block diagram of a safety warning judgment module in a production workshop construction operation safety warning system based on the Internet of Things according to an embodiment of the present application.

[0020] Figure 5 It is a flow chart of a production workshop construction operation safety early warning method based on the Internet of Things according to an embodiment of the present application.

[0021] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0023] Exemplary Systems

[0024] Figure 1 FIG. 1 is a block diagram of a production workshop construction safety early warning system based on the Internet of Things according to an embodiment of the present application. Figure 1 As shown, according to the Internet of Things-based production workshop construction safety warning system 100 of the embodiment of the present application, it includes: a construction vehicle driving data acquisition module 110, which is used to obtain the surrounding environment image of the construction vehicle during the driving process collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase-type laser ranging sensor; a construction vehicle driving data extraction module 120, which is used to extract the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle during the driving process collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase-type laser ranging sensor; a safety warning judgment module 130, which is used to judge whether to issue a safety warning alarm based on the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector.

[0025] In the above-mentioned production workshop construction safety early warning system 100 based on the Internet of Things, the construction vehicle driving data acquisition module 110 is used to obtain the surrounding environment image of the construction vehicle driving process collected by the camera and the distance measurement value of the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor. It should be understood that when performing workshop construction and maintenance operations, the key is to ensure that the construction unit does not change the position, structure and operation mode of the existing production equipment without interrupting production. The time and location of the construction task are usually uncertain, showing randomness and temporary nature. Considering the particularity of the workshop production environment, such as heavy load of driving, long-distance operation, irregular movement and noise problems, there are certain risks, such as the possibility of cargo falling, posing a threat to construction personnel. These factors require special attention to ensure that the construction is carried out safely while maintaining production continuity. Relying solely on the distance measurement value between the construction vehicle and the surrounding environment may lead to inaccurate early warnings. Therefore, in the technical solution of the present application, by combining deep learning technology, the surrounding environment images of the construction vehicle during driving are collected by the camera and the distance measurement values ​​of the construction vehicle and the surrounding environment collected by the phase laser ranging sensor at multiple predetermined time points to improve the accuracy of the safety warning of the construction vehicle. Among them, the camera provides visual information, which can capture real-time images of the surrounding environment of the construction vehicle, including pedestrians, obstacles, etc., and provide intuitive environmental perception data for the system. These image data can be analyzed and processed to extract the characteristic information of the surrounding environment of the construction vehicle. On the other hand, the phase laser ranging sensor provides high-precision distance information and can measure the distance value between the construction vehicle and the surrounding environment in real time. The collection of these distance values ​​can help the system accurately understand the distance between the construction vehicle and the surrounding obstacles, so as to timely discover potential collision risks or safety hazards. By comprehensively utilizing the data collected by the camera and the laser ranging sensor, the system can fully perceive the surrounding environment of the construction vehicle, realize all-round monitoring and safety assessment of the driving process of the construction vehicle, and issue safety warning alarms in time to help avoid potential accident risks and ensure the safe progress of construction work in the production workshop.

[0026] In the above-mentioned production workshop construction safety warning system 100 based on the Internet of Things, the construction vehicle driving data extraction module 120 is used to extract the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle driving process collected by the camera and the distance measurement values ​​of the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor. It should be understood that by comprehensively utilizing the camera image and laser ranging data, the system can comprehensively consider the information of the vehicle's surrounding environment and vehicle position, thereby extracting the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector. These feature vectors are the key basis for the safety warning system to determine whether to issue an alarm, and can help the system accurately and timely identify potential safety risks, ensure the smooth progress of construction operations and minimize the possibility of accidents.

[0027] Figure 2 FIG. 1 is a block diagram of a construction vehicle driving data extraction module in a production workshop construction safety early warning system based on the Internet of Things according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the construction vehicle driving data extraction module 120 includes: a construction vehicle driving environment data preprocessing unit 121, which is used to perform data preprocessing on the surrounding environment image of the construction vehicle driving process collected by the camera to obtain multiple construction vehicle surrounding environment feature maps; a construction vehicle driving environment feature extraction unit 122, which is used to extract features from the multiple construction vehicle surrounding environment feature maps to obtain the construction vehicle surrounding environment associated feature vector; a construction vehicle ranging feature extraction unit 123, which is used to extract features from the distance measurement values ​​of the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor to obtain the construction vehicle distance associated feature vector. It should be understood that in the scene of safety warning of construction work in the production workshop, the image collected by the camera may be affected by factors such as light and contain a lot of noise and interference. Through data preprocessing, these interferences can be effectively eliminated and the quality and clarity of the image can be improved.

[0028] Furthermore, the feature extraction process can transform complex image information into concise and meaningful feature representations. By extracting features from multiple construction vehicle surrounding environment feature maps, the system can dig deeper into the information in the image and obtain the associated feature vectors of the construction vehicle surrounding environment. These feature vectors contain key information in the image, such as obstacle locations, surrounding environment conditions, potential risk factors, etc., which helps the system to more accurately analyze and understand the situation around the vehicle and provide more accurate safety warnings and decision support.

[0029] Furthermore, by collecting the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points, meaningful features can be extracted from them so that the system can better understand and analyze the positional relationship of the vehicles, thereby achieving more accurate safety monitoring and early warning. Among them, feature extraction can help the system identify and distinguish different behavior patterns or situations. By extracting the characteristics of the vehicle's distance from the surrounding environment, the system can determine whether the vehicle is in a potentially dangerous situation, such as being too close to an obstacle or deviating from the lane. These features can serve as the basis for the system to conduct safety warnings and monitoring, and to promptly detect and respond to potential safety risks.

[0030] In a specific embodiment of the present application, the working vehicle driving environment data preprocessing unit 121 includes: dividing the surrounding environment image of the construction working vehicle driving process collected by the camera to obtain multiple construction working vehicle driving surrounding environment images; passing the multiple construction working vehicle driving surrounding environment images through the construction working vehicle surrounding environment convolution neural network model with multi-feature extraction capability to obtain the multiple construction working vehicle surrounding environment feature maps. It should be understood that by dividing the image of the entire driving process into multiple small areas or time periods, the various parts around the vehicle can be observed and analyzed more carefully, providing the system with more comprehensive and detailed information. This division can be based on space or time, for example, the entire driving section is divided into several areas or the driving process is divided into different time periods, so that the system can perform individual analysis for each small area or time period. In addition, the divided image can also help the system to realize real-time monitoring and tracking of the vehicle driving process. By dividing the entire driving process into multiple small areas or time periods, the system can capture changes in the vehicle's surrounding environment more promptly, so as to respond more quickly and provide early warning information to ensure driving safety.

[0031] Furthermore, multiple images of the surrounding environment of the construction vehicle are processed by a construction vehicle surrounding environment convolutional neural network model with multi-feature extraction capabilities, aiming to extract rich and meaningful feature information from these images. Among them, by processing multiple images of the surrounding environment of the construction vehicle by the construction vehicle surrounding environment convolutional neural network model, multiple feature maps of the surrounding environment of the construction vehicle can be obtained. These feature maps contain rich information learned and extracted by the model, such as feature representations of environmental elements such as roads and obstacles. Specifically, the construction vehicle surrounding environment convolutional neural network model with multi-feature extraction capabilities includes: a first convolutional layer, a second convolutional layer parallel to the first convolutional layer, and a cascade layer connected to the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional layer uses a one-dimensional convolutional kernel with a second scale.

[0032] Figure 3FIG. 1 is a block diagram of a vehicle driving environment feature extraction unit in a production workshop construction operation safety warning system based on the Internet of Things according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the construction vehicle driving environment feature extraction unit 122 includes: a construction vehicle environment target feature encoding subunit 1221, which is used to perform target feature encoding on the multiple construction vehicle surrounding environment feature maps to obtain a construction vehicle surrounding environment associated feature map; a construction vehicle environment pooling subunit 1222, which is used to perform global mean pooling on the construction vehicle surrounding environment associated feature map to obtain the construction vehicle surrounding environment associated feature vector. It should be understood that the correlation and association between different feature maps can be modeled and captured through target feature encoding. In this way, the connection and influence between various aspects of the construction vehicle surrounding environment can be better understood, the potential laws and information hidden in the feature map can be discovered, and the system can be provided with a deeper cognitive and analytical capability.

[0033] Furthermore, global mean pooling of the construction vehicle's surrounding environment-related feature maps can integrate and compress the rich information in these feature maps, thereby obtaining a compact and representative construction vehicle's surrounding environment-related feature vector. Among them, global mean pooling is a commonly used pooling operation. By calculating the average value of the entire feature map, the main features can be effectively retained and the feature dimension can be reduced. It should be understood that in the construction vehicle's surrounding environment-related feature map, each pixel contains the feature information of the corresponding position, and global mean pooling uniformly processes the information of the entire feature map, averages the feature values ​​of each channel, and obtains a global feature description. This operation helps to reduce feature dimensions and reduce computational complexity, while retaining key features, making the final construction vehicle's surrounding environment-related feature vector more representative and stable.

[0034] In a specific embodiment of the present application, the construction vehicle environment target feature encoding subunit 1221 includes: passing the multiple construction vehicle surrounding environment feature maps through the construction vehicle environment target detection network to obtain multiple construction vehicle surrounding environment interest regions; passing the multiple construction vehicle surrounding environment interest regions through the construction vehicle surrounding environment significant target detector to obtain the construction vehicle surrounding environment associated feature map. It should be understood that the construction vehicle environment target detection network can perform efficient feature extraction and target recognition on the input construction vehicle surrounding environment feature map through deep learning algorithms and target detection technology. In this way, the network can accurately capture the feature information of various targets in the construction vehicle surrounding environment, including pedestrians, obstacles, etc., so as to achieve comprehensive detection and recognition of the area of ​​interest. Accordingly, in a specific example of the present application, the construction vehicle environment target detection network is FastR-CNN, Faster R-CNN or Ret i naNet. Specifically, the construction vehicle environment target detection network is used to process the multiple construction vehicle surrounding environment feature maps with the following target detection formula to obtain the multiple construction vehicle surrounding environment interest regions;

[0035] Among them, the target detection formula is:

[0036]

[0037] Among them, ψ det is the feature map of the surrounding environment of multiple construction vehicles, B is the anchor frame, ROI is the region of interest of the surrounding environment of multiple construction vehicles, cls(ψ det ,B) indicates classification, Regr(ψ det ,B) indicates regression.

[0038] Furthermore, multiple regions of interest in the surrounding environment of the construction vehicle are processed by the salient target detector of the surrounding environment of the construction vehicle, aiming to extract key environmental features from these regions, and then generate a correlation feature map of the surrounding environment of the construction vehicle. The key to this process is that the salient target detector can more accurately identify and extract the most significant and important targets in the surrounding environment of the construction vehicle. Through the salient target detector of the surrounding environment of the construction vehicle, the most significant target information can be further extracted from the region of interest, that is, those targets that are more visually prominent and important, so as to accurately capture the key targets in the surrounding environment of the construction vehicle and help the system better understand and grasp the environmental features. Specifically, each layer of the construction vehicle surrounding environment salient target detector performs the following operations on the input data in the forward pass of the layer: convolution processing is performed on the input data using a first convolution kernel to obtain a first convolution feature map; convolution processing is performed on the first convolution feature map using a second convolution kernel to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than the size of the second convolution kernel; pooling processing is performed on the second convolution feature map to obtain a pooled feature map; and activation processing is performed on the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the construction vehicle surrounding environment salient target detector is the construction vehicle surrounding environment associated feature map, and the input of the first layer of the construction vehicle surrounding environment salient target detector is the multiple construction vehicle surrounding environment interest regions.

[0039] In a specific embodiment of the present application, the work vehicle ranging feature extraction unit 123 includes: constructing the ranging values ​​of the construction work vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor as a work vehicle environment distance input vector; passing the work vehicle environment distance input vector through the surrounding environment distance time series encoding model to obtain the work vehicle distance associated feature vector. It should be understood that by collecting the ranging values ​​of the construction work vehicle and the surrounding environment at multiple time points, the distance information of the work vehicle and the surrounding environment at different times can be obtained. This time-series data collection can capture the dynamic changes of the surrounding environment of the work vehicle, including the movement of the work vehicle, the change of the position of the surrounding obstacles, etc., and provide the system with richer and more accurate spatial information. In addition, constructing these ranging values ​​as the work vehicle environment distance input vector can integrate the distance information of multiple time points into a vector representation, simplifying the data processing and analysis process. Such a vector representation can better reflect the overall relationship between the work vehicle and the surrounding environment, and provide the system with more efficient and comprehensive input features.

[0040] Furthermore, through the surrounding environment distance temporal coding model, the time series information in the work vehicle environment distance input vector can be effectively encoded and modeled. Among them, the generation of the work vehicle distance correlation feature vector can help the system better understand the distance relationship between the work vehicle and the surrounding environment, and provide important information support for the work vehicle's autonomous navigation, obstacle avoidance and path planning tasks. Through the feature vector generated by the temporal coding model, the system can more accurately grasp the distance change trend between the work vehicle and the surrounding environment. Specifically, the work vehicle environment distance input vector is fully connected and encoded using the fully connected layer of the surrounding environment distance temporal coding model to extract the high-dimensional implicit features of the eigenvalues ​​at each position in the work vehicle environment distance input vector; and, the work vehicle environment distance input vector is one-dimensionally encoded using the one-dimensional convolutional layer of the surrounding environment distance temporal coding model to extract the high-dimensional implicit correlation features of the correlation between the eigenvalues ​​at each position in the work vehicle environment distance input vector.

[0041] In the above-mentioned production workshop construction safety warning system 100 based on the Internet of Things, the safety warning judgment module 130 is used to determine whether to issue a safety warning alarm based on the surrounding environment associated feature vector of the construction vehicle and the distance associated feature vector of the work vehicle. It should be understood that by analyzing the surrounding environment associated feature vector of the construction vehicle and the distance associated feature vector of the work vehicle, the system can detect the relationship between the work vehicle and the surrounding environment, including distance change, speed, direction and other information. If the system detects potential safety risks, such as the distance between the work vehicle and the obstacle is too close or the speed is abnormal, a safety warning alarm can be issued. This intelligent warning system can help monitoring personnel to detect potential dangerous situations in a timely manner, effectively reduce the possibility of accidents, and improve the safety and efficiency of the construction site. This intelligent safety monitoring system will bring more convenience and protection to the construction industry.

[0042] Figure 4 FIG. 1 is a block diagram of a safety warning judgment module in a production workshop construction operation safety warning system based on the Internet of Things according to an embodiment of the present application. Figure 4As shown, in a specific embodiment of the present application, the safety warning judgment module 130 includes: a work vehicle data feature fusion unit 131, which is used to fuse the surrounding environment associated feature vector of the construction work vehicle and the distance associated feature vector of the construction work vehicle to obtain a safety warning judgment feature vector; an optimization unit 132, which is used to calculate the equivalent strength representation of the safety warning judgment feature vector based on class probability to obtain an optimized safety warning judgment feature vector; and a warning alarm judgment generation unit 133, which is used to pass the optimized safety warning judgment feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a safety warning alarm. It should be understood that by fusing these two types of feature vectors, the system can understand the surrounding environment state of the work vehicle and the location information of the work vehicle itself at a higher level, thereby achieving a more comprehensive and accurate safety warning judgment. Among them, the surrounding environment associated feature vector of the construction work vehicle provides important information about the surrounding environment, including road conditions, obstacle locations, other vehicle conditions, etc., which are crucial for the safe driving of the work vehicle. The distance associated feature vector of the work vehicle reflects the distance relationship between the work vehicle and the surrounding environment, which can help the system accurately grasp the distance changes between the work vehicle and the surrounding objects. By fusing the feature vector associated with the surrounding environment of the construction vehicle and the feature vector associated with the distance of the construction vehicle, the complex relationship between the construction vehicle and the surrounding environment can be comprehensively considered, so as to more accurately judge the safety warning situation.

[0043] In particular, in the technical solution of the present application, first, an image of the surrounding environment of the construction vehicle is obtained from a camera, and a distance measurement value between the construction vehicle and the surrounding environment is obtained from a phase laser ranging sensor. The image will be segmented, and then a plurality of feature maps will be extracted through a convolutional neural network, and then a region of interest will be obtained through a target detection network, and then an associated feature map will be obtained through a significant target detector. At the same time, the ranging value is processed to construct a feature vector related to the distance of the working vehicle environment, and finally these feature vectors are fused into a safety warning judgment feature vector to predict whether a safety warning alarm will be issued. Considering that there may be class imbalance in the safety warning judgment task. This means that the number of samples of different categories (which may be "issuing a safety warning" and "not issuing a safety warning") is unbalanced. In the safety monitoring of construction vehicles, the situation of issuing a safety warning may be relatively rare, while the situation of normal driving without warning may be more common. This imbalance will affect the learning and prediction results of the model. Class imbalance may cause the model to be biased towards the majority category, thereby affecting the prediction ability of the minority category. In order to solve this problem, in the technical solution of the present application, the equivalent strength representation based on class probability of the safety warning judgment feature vector is calculated.

[0044] Among them, calculating the class probability-based equivalent strength representation of the safety warning judgment feature vector to obtain the optimized safety warning judgment feature vector includes: using the trained pre-classification model to perform classification regression on the safety warning judgment feature vector to obtain the class probability feature vector; calculating the inverse of the class probability value of each position in the class probability feature vector as the equivalent strength value to obtain the class probability equivalent strength representation feature vector; multiplying the class probability feature vector by the characteristic value of each position in the class probability equivalent strength representation feature vector as a coefficient to obtain a sequence of equivalent strength representation safety warning judgment feature vectors; and inputting the sequence of equivalent strength representation safety warning judgment feature vectors into an optimized encoder based on the RNN model to obtain the optimized safety warning judgment feature vector.

[0045] Here, the equivalent strength characterization based on class probability of the safety warning judgment feature vector is calculated to obtain the optimized safety warning judgment feature vector. First, the safety warning judgment feature vector to be optimized is input into the model for classification regression using the trained pre-classification model. The model will output the probability value of each category to form a class probability feature vector to provide information about the category to which the safety warning judgment feature vector belongs. Then, the reciprocal of the class probability value of each position in the class probability feature vector is taken, and each position of the class probability feature vector is multiplied accordingly, so as to convert the importance information of the class probability into an equivalent strength characterization to better represent the importance of the feature vector. At the same time, the optimization process can be more directly adjusted for the class probability of the feature vector, so as to better optimize the safety warning judgment feature vector. Finally, the sequence of the obtained equivalent strength characterization safety warning judgment feature vector is input into the optimization encoder based on the RNN model to further process these feature vector sequences and optimize them to obtain the final optimized safety warning judgment feature vector, so that the safety warning judgment feature vector better represents the importance of the feature and the category probability information, so as to help the model better distinguish the feature differences between different categories and improve the performance and generalization ability of the classification model. In addition, the equivalent strength representation based on class probability can better balance the importance between different categories and improve the interpretability of the model.

[0046] Specifically, the inverse of the class probability value at each position in the class probability feature vector is calculated as an equivalent strength value to obtain a class probability equivalent strength characterization feature vector, including: creating an aspect class for defining aspect logic; enabling AOP support in a Spring configuration file; creating a service class, wherein the service class contains a calculation method for calculating the inverse of the class probability value at each position in the class probability feature vector as an equivalent strength value to obtain a class probability equivalent strength characterization feature vector; and, calling the calculation method in the service class in the code to trigger the logic of the AOP aspect.

[0047] Here, part of the implementation code is as follows.

[0048]

[0049]

[0050] In this case, using the Spring AOP mechanism to implement "calculating the inverse of the class probability value at each position in the class probability feature vector as the equivalent strength value" has some advantages:

[0051] 1. Decoupling business logic and cross-cutting concerns: AOP can help separate cross-cutting concerns (such as calculating the inverse) from business logic (calculating the class probability feature vector), making the code more modular and maintainable.

[0052] 2. Reusability: Through AOP, you can apply the countdown calculation logic to multiple places without having to write code repeatedly, which improves the reusability of the code.

[0053] 3. Easy to maintain: Abstracting cross-cutting concerns into aspects makes it easier to manage and maintain the logic of these cross-cutting concerns without scattering them throughout the code base.

[0054] 4. Extensibility: If you need to modify the countdown calculation logic or add other similar cross-cutting concerns in the future, using AOP can make it easier to extend and modify.

[0055] 5. Code cleanliness: AOP can make business logic clearer and more concise, because the logic of cross-cutting concerns is separated into separate aspects and will not be mixed with business logic.

[0056] In general, using Spring AOP mechanism can help you better manage and organize your code, improve the maintainability, extensibility, and cleanliness of your code, while achieving the required functionality.

[0057] Furthermore, the classification results obtained by the classifier can be used to determine whether to issue a safety warning alarm. If the classification results indicate that there is a safety risk, the alarm system can be triggered immediately to remind relevant personnel to take appropriate measures to ensure the safety of construction vehicles and the surrounding environment. This automated safety warning system can greatly improve the safety and efficiency of construction sites, reduce the possibility of accidents, and protect the lives and property of workers.

[0058] In summary, the embodiment of the present application first obtains the surrounding environment image of the construction vehicle during its driving process captured by the camera and the ranging values ​​of the construction vehicle and the surrounding environment at multiple predetermined time points captured by the phase laser ranging sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, a classifier is used to obtain a classification result to determine whether to issue a safety warning alarm, thereby further improving the safety performance of the construction vehicle and reducing the risk of accidents.

[0059] As described above, the production workshop construction operation safety warning system 100 based on the Internet of Things according to the embodiment of the present application can be implemented in various terminal devices, such as a server deployed with a production workshop construction operation safety warning algorithm based on the Internet of Things. In one example, the production workshop construction operation safety warning system 100 based on the Internet of Things can be integrated into the terminal device as a software module and / or a hardware module. For example, the production workshop construction operation safety warning system 100 based on the Internet of Things can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the production workshop construction operation safety warning system 100 based on the Internet of Things can also be one of the many hardware modules of the terminal device.

[0060] Alternatively, in another example, the IoT-based production workshop construction operation safety warning system 100 and the terminal device may also be separate devices, and the IoT-based production workshop construction operation safety warning system 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0061] Exemplary Methods

[0062] Figure 5 Flow chart of the method for early warning of safety in production workshop construction operations based on the Internet of Things according to an embodiment of the present application. Figure 5 As shown, according to the Internet of Things-based production workshop construction safety warning method of the embodiment of the present application, it includes: S110, acquiring the surrounding environment image of the construction vehicle during the driving process collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase-type laser ranging sensor; S120, extracting the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle during the driving process collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase-type laser ranging sensor; S130, judging whether to issue a safety warning alarm based on the construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector.

[0063] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned production workshop construction operation safety early warning method based on the Internet of Things have been referred to above. Figures 1 to 4 The description of the IoT-based production workshop construction operation safety early warning system has been introduced in detail, and therefore, its repeated description will be omitted.

[0064] Exemplary Electronic Devices

[0065] Below, reference Figure 6 To describe an electronic device according to an embodiment of the present application.

[0066] like Figure 6 As shown, the electronic device 10 includes an input device 11, an input interface 12, a central processing unit 13, a memory 14, an output interface 15, an output device 16, and a bus 17. The input interface 12, the central processing unit 13, the memory 14, and the output interface 15 are connected to each other through the bus 17, and the input device 11 and the output device 16 are connected to the bus 17 through the input interface 12 and the output interface 15, respectively, and then connected to other components of the electronic device 10.

[0067] Specifically, the input device 11 receives input information from the outside and transmits the input information to the central processing unit 13 through the input interface 12; the central processing unit 13 processes the input information based on the computer executable instructions stored in the memory 14 to generate output information, temporarily or permanently stores the output information in the memory 14, and then transmits the output information to the output device 16 through the output interface 15; the output device 16 outputs the output information to the outside of the electronic device 10 for user use.

[0068] In one embodiment, Figure 6 The electronic device 10 shown can be implemented as a network device, which may include: a memory configured to store programs; a processor configured to run the programs stored in the memory to execute any one of the Internet of Things-based production workshop construction work safety warning methods described in the above embodiments.

[0069] According to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable storage medium.

[0070] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0071] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A production workshop construction safety early warning system based on the Internet of Things, characterized in that: include: A construction vehicle driving data acquisition module is used to acquire the surrounding environment image of the construction vehicle during driving collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor; A construction vehicle driving data extraction module is used to extract a construction vehicle surrounding environment associated feature vector and a construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle during driving collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor; A safety warning judgment module, used to judge whether to issue a safety warning alarm based on the surrounding environment associated feature vector of the construction vehicle and the distance associated feature vector of the construction vehicle; Wherein, the safety warning judgment module includes: A work vehicle data feature fusion unit, used for fusing the construction vehicle surrounding environment associated feature vector and the work vehicle distance associated feature vector to obtain a safety warning judgment feature vector; An optimization unit, used for calculating an equivalent strength representation of the safety warning judgment feature vector based on class probability to obtain an optimized safety warning judgment feature vector; A warning alarm judgment generating unit, used for passing the optimized safety warning judgment feature vector through a classifier to obtain a classification result, wherein the classification result is used to determine whether to issue a safety warning alarm; Wherein, the optimization unit comprises: Using the trained pre-classification model, the safety warning judgment feature vector is classified and regressed to obtain a class probability feature vector; Calculating the inverse of the class probability value at each position in the class probability feature vector as an equivalent strength value to obtain a class probability equivalent strength characterization feature vector; Using the eigenvalues ​​of each position in the class probability equivalent strength characterization feature vector as coefficients, multiplying the class probability feature vector respectively to obtain a sequence of equivalent strength characterization safety warning judgment feature vectors; and The sequence of the equivalent strength characterization safety warning judgment feature vector is input into an optimized encoder based on the RNN model to obtain the optimized safety warning judgment feature vector.

2. According to the Internet of Things-based production workshop construction safety early warning system according to claim 1, it is characterized in that: The construction vehicle driving data extraction module comprises: A construction vehicle driving environment data preprocessing unit, used for performing data preprocessing on the surrounding environment images of the construction vehicle during driving collected by the camera to obtain a plurality of construction vehicle surrounding environment feature maps; A construction vehicle driving environment feature extraction unit, used for extracting features from the plurality of construction vehicle surrounding environment feature maps to obtain a construction vehicle surrounding environment associated feature vector; The working vehicle distance measurement feature extraction unit is used to extract features from the distance measurement values ​​of the construction working vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor to obtain the working vehicle distance associated feature vector.

3. The production workshop construction operation safety early warning system based on the Internet of Things according to claim 2 is characterized in that: The working vehicle driving environment data preprocessing unit comprises: Dividing the surrounding environment images of the construction vehicle during its driving process collected by the camera to obtain a plurality of surrounding environment images of the construction vehicle during its driving process; The images of the multiple construction vehicles' surrounding environments are passed through a construction vehicle's surrounding environment convolutional neural network model with multi-feature extraction capabilities to obtain feature maps of the multiple construction vehicles' surrounding environments.

4. The production workshop construction operation safety early warning system based on the Internet of Things according to claim 3 is characterized in that: The working vehicle driving environment feature extraction unit comprises: A construction vehicle environment target feature encoding subunit, used for performing target feature encoding on the plurality of construction vehicle surrounding environment feature maps to obtain a construction vehicle surrounding environment associated feature map; The construction vehicle environment pooling subunit is used to perform global mean pooling on the construction vehicle surrounding environment associated feature map to obtain the construction vehicle surrounding environment associated feature vector.

5. The production workshop construction operation safety early warning system based on the Internet of Things according to claim 4 is characterized in that: The working vehicle environment target feature encoding subunit includes: Passing the plurality of construction vehicle surrounding environment feature maps through a construction vehicle environment target detection network to obtain a plurality of construction vehicle surrounding environment interest regions; The multiple construction vehicle surrounding environment interest regions are passed through a construction vehicle surrounding environment salient target detector to obtain a construction vehicle surrounding environment associated feature map.

6. The IoT-based production workshop construction safety early warning system according to claim 5 is characterized in that: The working vehicle ranging feature extraction unit comprises: Constructing the distance values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor as a construction vehicle environment distance input vector; The working vehicle environment distance input vector is passed through the surrounding environment distance temporal coding model to obtain the working vehicle distance associated feature vector.

7. The IoT-based production workshop construction safety early warning system according to claim 6 is characterized in that: Calculating the inverse of the class probability value at each position in the class probability feature vector as an equivalent strength value to obtain a class probability equivalent strength characterization feature vector, including: Create an aspect class to define the aspect logic; Enable AOP support in Spring's configuration file; Creating a service class, wherein the service class includes a calculation method for calculating the inverse of the class probability value at each position in the class probability feature vector as an equivalent strength value to obtain a class probability equivalent strength characterization feature vector; and The calculation method in the service class is called in the code to trigger the logic of the AOP aspect.

8. A safety early warning method for production workshop construction operations based on the Internet of Things, characterized in that: include: Acquire the surrounding environment image of the construction vehicle during its driving process collected by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points collected by the phase laser ranging sensor; Extracting a construction vehicle surrounding environment associated feature vector and a construction vehicle distance associated feature vector from the surrounding environment image of the construction vehicle during its driving process acquired by the camera and the distance measurement values ​​between the construction vehicle and the surrounding environment at multiple predetermined time points acquired by the phase laser ranging sensor; Based on the feature vector associated with the surrounding environment of the construction vehicle and the feature vector associated with the distance of the construction vehicle, determining whether to issue a safety warning alarm; Wherein, judging whether to issue a safety warning alarm based on the surrounding environment associated feature vector of the construction vehicle and the distance associated feature vector of the construction vehicle includes: The construction vehicle surrounding environment associated feature vector and the construction vehicle distance associated feature vector are merged to obtain a safety warning judgment feature vector; Calculating the equivalent strength representation of the safety warning judgment feature vector based on class probability to obtain an optimized safety warning judgment feature vector; Passing the optimized safety warning judgment feature vector through a classifier to obtain a classification result, wherein the classification result is used to determine whether to issue a safety warning alarm; The step of calculating the equivalent strength representation of the safety warning judgment feature vector based on class probability to obtain the optimized safety warning judgment feature vector includes: Using the trained pre-classification model, the safety warning judgment feature vector is classified and regressed to obtain a class probability feature vector; Calculating the inverse of the class probability value at each position in the class probability feature vector as an equivalent strength value to obtain a class probability equivalent strength characterization feature vector; Using the eigenvalues ​​of each position in the class probability equivalent strength characterization feature vector as coefficients, multiplying the class probability feature vector respectively to obtain a sequence of equivalent strength characterization safety warning judgment feature vectors; and The sequence of the equivalent strength characterization safety warning judgment feature vector is input into an optimized encoder based on the RNN model to obtain the optimized safety warning judgment feature vector.

Images