Anti-collision early warning method and system of crane, electronic equipment and storage medium

Abstract

The invention discloses an anti-collision early warning method and system for a crane, electronic equipment and a storage medium. Historical video monitoring data in the surrounding environment of the crane are imported into a lightweight network structure model for training; real-time monitoring data in the surrounding environment of the crane are imported into the trained target detection model to be recognized, and target information in the surrounding environment of the crane is obtained; the target information is imported into the binocular vision ranging algorithm model, and the real-time distance between the target object and the crane is obtained; and whether the real-time distance is smaller than a preset threshold value or not is judged, and if yes, corresponding early warning processing is conducted on the crane. Compared with the prior art, the method has the following effects: by designing the lightweight target detection model and the binocular vision ranging algorithm, the accuracy and efficiency of identifying the target around the crane can be improved, so that the target information and the real-time distance information can be obtained more accurately; therefore, the anti-collision safety early warning performance of the crane is improved.

Description

Technical Field

[0001] The present application relates to the technical field of target detection, and in particular to an anti-collision warning method, system, electronic equipment and storage medium for a crane. Background Art

[0002] Crane is a common marine lifting equipment, widely used in ports, docks and ships for cargo loading and unloading operations. It is mainly composed of a boom, a slewing mechanism, a balancer, a crane frame and an electrical control system, and various actions are achieved through a hydraulic system drive. The operation process of a crane includes starting the crane, lifting objects, moving objects, unloading objects and shutting down the crane. Its stability and safety are crucial to ensuring loading and unloading operations. At present, with the rapid development of my country's economy, there are more and more lifting operations, and cranes are used more widely. Therefore, the anti-collision safety of cranes is also given more attention.

[0003] At present, the existing crane anti-collision method is to record the scene through video monitoring technology, and then let the operator analyze whether there is a possibility of collision based on the shooting picture, and cooperate with the sensor to detect the distance between the crane and the obstacle, so as to control the crane to operate and achieve the effect of anti-collision. However, manual observation is prone to labor fatigue, leading to misoperation or misjudgment, which poses a greater risk. In addition, if the crane is braked when it is judged that there is a risk, the collision may not be prevented in time due to the timeliness of the crane's braking, which further leads to a greater collision risk. Summary of the invention

[0004] The purpose of the present application is to provide a method, system, electronic device and storage medium for preventing collision of a crane, so as to solve the problem in the related art that it is not possible to accurately and timely provide an early warning of the collision of the crane.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a crane anti-collision warning method, comprising:

[0007] The method comprises the following steps: obtaining historical video surveillance data in the environment around the crane, and preprocessing the historical video surveillance data to obtain preprocessed historical video surveillance data; importing the preprocessed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the environment around the crane; obtaining real-time video surveillance data in the environment around the crane, and importing the real-time surveillance data into the target detection model for identification and detection to obtain target information in the environment around the crane, wherein the target information includes: category information and position information of the target; importing the target information into a preset binocular vision ranging algorithm model to obtain a real-time distance between the target object and the crane; judging whether the real-time distance is less than a preset threshold value, and if so, performing corresponding early warning processing on the crane.

[0008] Preferably, the acquiring of historical video surveillance data in the environment around the crane and preprocessing the historical video surveillance data to obtain preprocessed historical video surveillance data comprises: acquiring historical video surveillance data in the environment around the crane and performing frame extraction processing on the historical video surveillance data to obtain a historical surveillance image dataset in the environment around the crane; and performing image scale normalization and image standardization processing on each frame of the historical surveillance image dataset to obtain a processed historical surveillance image dataset in the environment around the crane.

[0009] Preferably, the preset lightweight network structure model is specifically constructed in the following steps: designing a loss function, the loss function including a positioning loss function and a similarity measurement loss function; and constructing a lightweight network structure model according to the loss function and the YOLOv8 algorithm.

[0010] Preferably, the loss function is calculated as follows:

[0011]

[0012] Among them, F loss is the loss function, L loc is the positioning loss function, L cls is the similarity measurement loss function, i is the index number of the anchor point of the i-th candidate region in the image, and p i Represents the target prediction probability of the anchor point; represents the candidate region label, N loc With N cls Respectively represent the classification loss function in the positioning loss L loc With the similarity metric loss L cls Normalized coefficient after functionalization, λ represents N loc With N cls The balance weight between .

[0013] Preferably, the positioning loss function L loc , the calculation formula is as follows:

[0014] L loc =αL reg +βL CIoU

[0015] Where L reg represents the position regression loss function, L CIoU is the generalized intersection-over-union loss function, and α and β are hyperparameters used to adjust the functions of each part;

[0016] The position regression loss function L reg The calculation formula is as follows:

[0017]

[0018] Among them, B is the predicted box, B′ is the target box, IoU represents the ratio of the intersection and union of the target box and the real box, and B C represents the smallest enclosed box in the positional relationship between the prediction box and the target box, and U represents the union of the prediction box and the target box;

[0019] The generalized intersection-over-union loss function L CIoU The calculation formula is as follows:

[0020]

[0021] Among them, a is the weight parameter; v is used to measure the consistency of aspect ratio, w′ and h′ are the width and height of the target box, respectively, and w and h are the width and height of the prediction box, respectively;

[0022] L cls Represents the similarity measurement loss function, and the calculation formula is as follows:

[0023]

[0024] Among them, x represents the true sample label, and x′ represents the detection output after the activation function.

[0025] Preferably, the pre-processed historical video surveillance data is imported into a preset lightweight network structure model for training to obtain a target detection model in the surrounding environment of the crane, including:

[0026] The historical video surveillance data is divided into a training set, a validation set and a test set; the training set is subjected to data enhancement processing to obtain an enhanced training set; the enhanced training set and the validation set are imported into a preset lightweight network structure model for training, and a target detection model in the environment around the crane is obtained through training.

[0027] Preferably, an anti-collision warning is given to the crane according to the real-time distance, including: when the real-time distance is less than or equal to a first set distance, a voice warning is given; when the real-time distance is less than or equal to a second set distance, the crane is controlled to stop running.

[0028] In a second aspect, the present application further provides an anti-collision warning system for a crane, comprising:

[0029] An acquisition module is used to acquire historical video surveillance data in the environment around the crane, and pre-process the historical video surveillance data to obtain the pre-processed historical video surveillance data; a training module is used to import the pre-processed historical video surveillance data into a preset lightweight network structure model for training, and obtain a target detection model in the environment around the crane; an identification and detection module is used to acquire real-time video surveillance data in the environment around the crane, and import the real-time surveillance data into the target detection model for identification and detection, and obtain target information in the environment around the crane, wherein the target information includes: target category information and position information; a ranging module is used to import the target information into a preset binocular vision ranging algorithm model to obtain the real-time distance between the target object and the crane; an early warning module is used to determine whether the real-time distance is less than a preset threshold, and if so, perform corresponding early warning processing on the crane.

[0030] In a third aspect, the present application further provides a computer electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the anti-collision warning method for a crane as described in any one of the above are implemented.

[0031] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the anti-collision warning method for a crane as described in any one of the above are implemented.

[0032] The present application provides a method, system, electronic device and storage medium for anti-collision warning of a crane. The method comprises the following steps: obtaining historical video surveillance data in the environment around the crane, and preprocessing the historical video surveillance data to obtain preprocessed historical video surveillance data; importing the preprocessed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the environment around the crane; obtaining real-time video surveillance data in the environment around the crane, and importing the real-time surveillance data into the target detection model for identification and detection to obtain target information in the environment around the crane, wherein the target information includes: category information and location information of the target; importing the target information into a preset binocular vision ranging algorithm model to obtain the real-time distance between the target object and the crane; judging whether the real-time distance is less than a preset threshold value, and if so, performing corresponding warning processing on the crane. Compared with the prior art, the present invention has the following effects: by designing a lightweight target detection model and a binocular vision ranging algorithm, the present invention can improve the accuracy and efficiency of identifying targets around the crane, thereby more accurately acquiring target information and real-time distance information, thereby improving the performance of the crane's anti-collision safety warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of a method for preventing collision and early warning of a crane according to an embodiment of the present application;

[0034] Figure 2 It is a flow chart of the binocular vision ranging algorithm in the embodiment of the present application;

[0035] Figure 3 It is a structural diagram of a lightweight network structure model in an embodiment of the present application;

[0036] Figure 4 It is a structural schematic diagram of a multi-scale decoupled head in a lightweight network structure model in an embodiment of the present application;

[0037] Figure 5 It is a structural schematic diagram of an anti-collision warning system for a crane according to an embodiment of the present application;

[0038] Figure 6 It is a structural schematic diagram of a computer electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0041] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0042] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0043] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms "a", "said" and "the" used in one or more embodiments of the present application are also intended to include plural forms, unless the context clearly indicates other meanings.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0045] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when the time of".

[0046] Please refer to Figure 1 , an embodiment of the present application provides an anti-collision warning method for a crane, the warning method being applied to a shipbuilding crane, comprising the following steps:

[0047] S10, acquiring historical video surveillance data of the surrounding environment of the crane, and preprocessing the historical video surveillance data to obtain preprocessed historical video surveillance data.

[0048] It is understandable that, due to the large working range of the crane, the working scene of the crane is generally filled with multiple video surveillance devices, and the historical data of these video surveillance devices are used as raw data for training and testing.

[0049] It should be noted that in order to improve the accuracy and effectiveness of the original data, the historical data needs to be preprocessed first. The preprocessing of the data may include: Data cleaning: including removing duplicate values, processing missing values, processing outliers, etc. to ensure the accuracy and consistency of the data; Data conversion: converting the data to adapt to specific analysis methods. This includes normalization / standardization, discretization and data transformation, etc.; Data transformation: mathematical transformation of the data to change the distribution of the data or reduce noise; Outlier processing: detecting and processing outliers to prevent them from having adverse effects on the data analysis results; Data smoothing: smoothing the data to eliminate noise and irregular changes.

[0050] S20, importing the pre-processed historical video surveillance data into a preset lightweight network structure model for training, and obtaining a target detection model in the surrounding environment of the crane.

[0051] Specifically, the data monitoring data preprocessed in step S10 is imported into a pre-built lightweight model for training, so as to obtain a trained target detection model in the surrounding environment of the crane.

[0052] It should be noted that during training, different training strategies can be selected for training, such as setting a training data budget and data enhancement operations. Setting a training data budget means that when starting a new machine learning project, you must first define your goals, which will determine what type of data is needed and how many "training items" (classified data points) are needed. For example, for a computer vision or image recognition project, you may need to use manually annotated image data for training. Depending on the needs of the project, the model may need to be constantly retrained or updated, so it is necessary to evaluate the options for procuring data and calculate a budget. Data enhancement refers to increasing the diversity of data through technical means to improve the generalization ability of the model. This includes operations such as flipping, cropping, rotating images, or transforming pixel colors, such as adding noise.

[0053] S30, acquiring real-time video surveillance data in the surrounding environment of the crane, and importing the real-time surveillance data into the target detection model for identification and detection, so as to obtain target information in the surrounding environment of the crane, wherein the target information includes: category information and location information of the target.

[0054] Specifically, after the target detection model is trained in step S20, real-time monitoring data from the video surveillance equipment around the crane can be collected, and the real-time monitoring data can be processed and imported into the target detection model for identification and detection, thereby obtaining target information in the environment around the crane.

[0055] It should be noted that the process of preprocessing the real-time monitoring data may refer to the description of preprocessing the historical monitoring data.

[0056] S40, importing the target information into a preset binocular vision ranging algorithm model to obtain a real-time distance between the target object and the crane.

[0057] Specifically, the target information around the crane is imported into a preset binocular vision algorithm model, so that the real-time distance between the target object and the crane can be obtained.

[0058] It should be noted that the binocular vision ranging algorithm is mainly implemented based on the similar triangle principle and the pixel scale principle. These two principles explain how to measure the distance of objects in the scene through two cameras (i.e. binocular cameras) from different perspectives. Similar triangle principle: It uses two cameras (Or and Ot) placed in parallel, the distance between the cameras is B, the focal length of each camera lens is f, a point in the scene is P, and the distance from the point to the camera is Z. Through the principle of similar triangles, the expression of Z can be derived to calculate the distance between the object and the camera. Pixel scale principle: The same scene is photographed by the camera at different positions, and the distance is calculated by the distance moved by the camera and the pixel change of the object in the captured image. Specifically, through the pixel coordinate change of the object in the image captured by the camera at position 1 and position 2, combined with the camera's field of view and lens parameters, the actual distance represented by each pixel can be calculated, and then the distance between the object and the camera can be obtained.

[0059] For example, please refer to Figure 2 ,The workflow of the binocular vision ranging algorithm is as follows:

[0060] First, after building the binocular vision ranging model, complete the calibration and verification and other preparation work;

[0061] Secondly, the left and right images acquired by the binocular camera are corrected using the binocular camera parameters, and the disparity maps of the corrected left and right images are obtained through the BM stereo matching algorithm;

[0062] Then, input the target location information, use the disparity information to calculate the distance of each point in the detection target location area and take the average value to calculate the distance D of the target;

[0063] Finally, the real-time distance between the target object and the crane in the detected image is output.

[0064] S50: determining whether the real-time distance is less than a preset threshold value; if so, performing corresponding early warning processing on the crane.

[0065] Specifically, when the real-time distance between the target objects around the crane and the crane is less than a preset threshold, corresponding warning processing can be performed on the crane, for example, slowing down the speed of the crane, sending warning information, and stopping the operation of the crane.

[0066] In some optional embodiments, when the real-time distance is less than or equal to a first set distance, a voice warning is issued; when the real-time distance is less than or equal to a second set distance, the crane is controlled to stop running.

[0067] Exemplarily, when the distance between the target object and the crane is less than or equal to 10 meters, a voice warning message can be sent to remind relevant personnel to handle the situation. When the distance between the target object and the crane is less than or equal to 3 meters, the crane is directly controlled to stop running to avoid a collision.

[0068] The present application provides a method for anti-collision warning of a crane, which is applied to the use scenario of the crane. The warning method obtains historical video surveillance data in the environment around the crane, and pre-processes the historical video surveillance data to obtain the pre-processed historical video surveillance data; imports the pre-processed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the environment around the crane; obtains real-time video surveillance data in the environment around the crane, and imports the real-time monitoring data into the target detection model for identification and detection to obtain target information in the environment around the crane, wherein the target information includes: category information and location information of the target; imports the target information into a preset binocular vision ranging algorithm model to obtain the real-time distance between the target object and the crane; and performs anti-collision warning on the crane according to the real-time distance. Compared with the prior art, the present application has the following effects: by designing a lightweight target detection model and a binocular vision ranging algorithm, the present application can improve the accuracy and efficiency of identifying targets around the crane, and then can more accurately obtain target information and real-time distance information, thereby improving the performance of anti-collision safety warning of the crane.

[0069] In some optional embodiments, step S10, obtaining historical video surveillance data of the surrounding environment of the crane and preprocessing the historical video surveillance data to obtain the preprocessed historical video surveillance data, comprises the following steps:

[0070] S11, acquiring historical video surveillance data of the environment around the crane, and performing frame extraction processing on the historical video surveillance data to obtain a historical surveillance image data set of the environment around the crane.

[0071] Specifically, after acquiring the historical video surveillance data, the video surveillance data first needs to be processed by frame extraction. The purpose of this is to extract key frames from the video data. These key frames usually contain the main information of the video, which can significantly reduce the amount of data that needs to be processed, thereby improving the efficiency of image processing.

[0072] It should be noted that the frame extraction method includes uniform frame extraction and specified time frame extraction. The former extracts the picture by setting a fixed time interval, and the latter extracts the picture according to a specific time range and number of frames. In this application, no specific limitation is made to the frame extraction method.

[0073] S12, performing image scale normalization and image standardization processing on each frame of the historical monitoring image data set to obtain a processed historical monitoring image data set of the crane's surrounding environment.

[0074] Specifically, after obtaining the historical monitoring image dataset of the crane's surrounding environment, it is necessary to perform image scale normalization and image standardization processing on each frame of the image dataset. Such processing can achieve the following effects:

[0075] Image scale normalization can adjust the size of the image to a unified standard. By adjusting images of different sizes to the same size, the consistency and comparability of the algorithm can be ensured. In addition, scale normalization also helps to reduce the consumption of computing resources, because images of different sizes may require different computing resources when processing. Through normalization, it can ensure that the algorithm has the same computational complexity when processing images of different sizes, thereby improving processing efficiency.

[0076] Image normalization can adjust the pixel values ​​of an image to a uniform range, such as [0,1] or [-1,1], which helps improve the numerical stability and convergence speed of the algorithm. Normalization can also make the pixel values ​​between different images comparable, so that the algorithm can more accurately extract the features of the image. In addition, normalization can also improve the generalization ability of the model, making it better adapt to different data distributions and reducing the risk of overfitting.

[0077] In this embodiment, by performing frame extraction, scale normalization, image standardization and other processing on the video image, the image processing efficiency can be improved, the consumption of computing resources can be reduced, and the model performance and stability can be improved.

[0078] In some optional embodiments, the preset lightweight network structure model is specifically constructed in the following steps: designing a loss function, the loss function including a positioning loss function and a similarity measurement loss function; and constructing a lightweight network structure model based on the loss function and the YOLOv8 algorithm.

[0079] Specifically, the specific structure of the lightweight network structure model is as follows Figure 3 As shown, the working principle of the lightweight network structure model is as follows:

[0080] In the first step, the monitoring stream is processed by the data enhancement module and then downsampled through a Focus layer.

[0081] In the second step, the image is passed through the data enhancement module and the Focus layer and then input into the backbone network for feature extraction to obtain three feature maps of different sizes.

[0082] It should be noted that the data enhancement module uses the mean shift filter algorithm (Mean Shift) and the contrast-limited adaptive histogram equalization algorithm (CLAHE) to preprocess the image. These two methods can effectively reduce background noise and enhance the information of crack edges.

[0083] The backbone network consists of a re-parameterization module, SC2f lightweight convolution, and SPPF module. The re-parameterization module and SC2f lightweight convolution are used to perform operations such as convolution, batch normalization, and pooling. Mish function activation is applied after batch normalization to reduce the number of parameters and optimize calculations. The feature pyramid module (SPPF) performs multiple pooling on feature maps, and then fuses and splices feature maps of multiple dimensions, so that the network can extract higher-level semantic features.

[0084] The third step is to input different feature maps into the attention mechanism module respectively, assign attention weights, and obtain new feature maps.

[0085] Specifically, the attention mechanism module consists of the Efficient Channel Attention (ECA) mechanism, which is a cross-channel attention mechanism without dimensionality reduction, which is conducive to improving the learning ability of the network model. To further improve the performance of the model, the input layer of ECA is subjected to adaptive average pooling and adaptive maximum pooling operations, followed by addition operations to enhance the global visual information of the module, so that the model can adapt to inputs of different sizes.

[0086] The fourth step is to input the new feature map into the multi-scale feature fusion module for feature fusion.

[0087] Specifically, the multi-scale feature fusion module first records the feature layer of size 20×20 output by the SPPF module as the X1 layer. Then, the X1 feature layer is upsampled by 2 times and concatenated with the feature layer of size 40×40 to form a concatenated 40×40 feature layer, which is recorded as the X2 feature layer. Next, the X2 feature layer and the X1 feature layer are upsampled by 2 times and 4 times, respectively, and concatenated with the feature layer of size 80×80 to obtain the X3 feature layer. Finally, in order to speed up the calculation speed of the network, the Hi module is added to each output layer to reduce the number of feature maps, Hi(i=1,2,3…)=Conv+BN+Mish. Finally, the X1 feature layer, X2 feature layer, and X3 feature layer output head classifiers for detecting small targets, medium targets, and large targets, respectively.

[0088] In the fifth step, through the small target decoupling head, the medium target decoupling head and the large target decoupling head processing, the information generated by the multi-scale decoupling head undergoes corresponding post-processing steps to form the final detection result and then output the result.

[0089] It should be noted that the multi-scale decoupled head structure is Figure 4 As shown, in Figure 4 In the figure, Reg, Obj and Cls are the prediction results. Reg is used to represent the regression parameters of each feature point. Obj and Cls are used to determine whether an object is included and the type of the object.

[0090] In some optional embodiments, the loss function is calculated as follows:

[0091]

[0092] Among them, F loss is the loss function, L loc is the positioning loss function, L cls is the similarity measurement loss function, i is the index number of the anchor point of the i-th candidate region in the image, and p i Represents the target prediction probability of the anchor point; represents the candidate region label, N loc With N cls Respectively represent the classification loss function in the positioning loss L loc With the similarity metric loss L cls Normalized coefficient after functionalization, λ represents N loc With N cls The balance weight between .

[0093] In some optional embodiments, the positioning loss function L loc , the calculation formula is as follows:

[0094] L loc =αL reg +βL CIoU

[0095] Where L reg represents the position regression loss function, L CIoU is the generalized intersection-over-union loss function, and α and β are hyperparameters used to adjust the functions of each part;

[0096] The position regression loss function L reg The calculation formula is as follows:

[0097]

[0098] Among them, B is the predicted box, B′ is the target box, IoU represents the ratio of the intersection and union of the target box and the real box, and B C represents the smallest enclosed box in the positional relationship between the prediction box and the target box, and U represents the union of the prediction box and the target box;

[0099] The generalized intersection-over-union loss function L CIoU The calculation formula is as follows:

[0100]

[0101] Among them, a is the weight parameter; v is used to measure the consistency of aspect ratio, w′ and h′ are the width and height of the target box, respectively, and w and h are the width and height of the prediction box, respectively;

[0102] L cls Represents the similarity measurement loss function, and the calculation formula is as follows:

[0103]

[0104] Among them, x represents the true sample label, and x′ represents the detection output after the activation function.

[0105] In some optional embodiments, step S20, importing the pre-processed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the crane surrounding environment, includes:

[0106] S21, dividing the historical video surveillance data into a training set, a validation set and a test set.

[0107] Exemplarily, the data in the historical video surveillance data is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0108] S22. Perform data enhancement processing on the training set to obtain an enhanced training set.

[0109] Exemplarily, the Mosaic and MixUp data enhancement methods can be used to enrich the training set;

[0110] It should be noted that the Mosaic data augmentation method first crops four randomly selected images and then splices them into a new image, while the MixUp data augmentation method mixes and superimposes two randomly selected images to form a new image.

[0111] S23, importing the enhanced training set and the verification set into a preset lightweight network structure model for training, so as to obtain a target detection model in the surrounding environment of the crane.

[0112] Exemplarily, the lightweight network structure is preliminarily trained using a training set, a validation set, and a loss function; the model weight parameters at the end of training are saved; the saved model weight parameters are tested using a test set, the initial accuracy and model inference speed are recorded, and a target detection model is obtained after training.

[0113] See also Figure 5The present application also provides a crane anti-collision warning system 200, comprising:

[0114] An acquisition module 201 is used to acquire historical video surveillance data in the surrounding environment of the crane, and pre-process the historical video surveillance data to obtain pre-processed historical video surveillance data;

[0115] A training module 202 is used to import the pre-processed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the surrounding environment of the crane;

[0116] The recognition and detection module 203 is used to obtain real-time video monitoring data in the surrounding environment of the crane, and import the real-time monitoring data into the target detection model for recognition and detection to obtain target information in the surrounding environment of the crane, wherein the target information includes: category information and location information of the target;

[0117] The distance measurement module 204 is used to import the target information into a preset binocular vision distance measurement algorithm model to obtain the real-time distance between the target object and the crane;

[0118] The early warning module 205 is used to determine whether the real-time distance is less than a preset threshold value, and if so, perform corresponding early warning processing on the crane.

[0119] See also Figure 6 The embodiment of the present application further provides a computer electronic device 400, the electronic device 400 includes a memory 403 and a processor 402, the memory 403 stores a computer program, and the processor implements the steps of any of the above-mentioned anti-collision warning methods for cranes when executing the computer program.

[0120] Specifically, the electronic device 400 includes: a transceiver 401, a bus interface and a processor 402, wherein the processor 402 is used to obtain historical video surveillance data in the environment around the crane, and pre-process the historical video surveillance data to obtain pre-processed historical video surveillance data; import the pre-processed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the environment around the crane; obtain real-time video surveillance data in the environment around the crane, and import the real-time monitoring data into the target detection model for identification and detection to obtain target information in the environment around the crane, wherein the target information includes: category information and location information of the target; import the target information into a preset binocular vision ranging algorithm model to obtain the real-time distance between the target object and the crane; determine whether the real-time distance is less than a preset threshold, and if so, perform corresponding early warning processing on the crane.

[0121] In an embodiment, the electronic device 400 further includes a memory 403. Figure 6 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 402 and memory represented by memory 403. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 401 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 402 is responsible for managing the bus architecture and general processing, and the memory 403 may store data used by the processor 402 when performing operations.

[0122] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method for distributing Chinese herbal medicines as described in any one of the above are implemented.

[0123] In this embodiment, the computer readable storage medium may be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include but is not limited to: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0124] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0125] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0126] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0127] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0128] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0129] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A crane anti-collision warning method, characterized in that: include: Acquire historical video surveillance data of the surrounding environment of the crane, and preprocess the historical video surveillance data to obtain preprocessed historical video surveillance data; Importing the preprocessed historical video surveillance data into a preset lightweight network structure model for training to obtain a target detection model in the crane surrounding environment; Acquire real-time video surveillance data in the surrounding environment of the crane, and import the real-time surveillance data into the target detection model for identification and detection, so as to obtain target information in the surrounding environment of the crane, wherein the target information includes: category information and location information of the target; Importing the target information into a preset binocular vision ranging algorithm model to obtain the real-time distance between the target object and the crane; It is determined whether the real-time distance is less than a preset threshold value. If so, a corresponding early warning process is performed on the crane.

2. The anti-collision warning method for a crane according to claim 1, characterized in that: The step of acquiring historical video surveillance data of the surrounding environment of the crane and preprocessing the historical video surveillance data to obtain the preprocessed historical video surveillance data includes: Acquire historical video surveillance data in the surrounding environment of the crane, and perform frame extraction processing on the historical video surveillance data to obtain a historical surveillance image data set in the surrounding environment of the crane; Image scale normalization and image standardization processing are performed on each frame of the historical monitoring image data set to obtain a processed historical monitoring image data set of the crane surrounding environment.

3. The anti-collision warning method for a crane according to claim 1, characterized in that: The specific construction steps of the preset lightweight network structure model are as follows: Designing a loss function, wherein the loss function includes a positioning loss function and a similarity measurement loss function; According to the loss function and YOLOv8 algorithm, a lightweight network structure model is constructed.

4. The anti-collision warning method for a crane according to claim 3, characterized in that: The loss function is calculated as follows: Among them, F loss is the loss function, L loc is the positioning loss function, L cls is the similarity measurement loss function, i is the index number of the anchor point of the i-th candidate region in the image, and p i Represents the target prediction probability of the anchor point; represents the candidate region label, N loc With N cls Respectively represent the classification loss function in the positioning loss L loc With the similarity metric loss L cls Normalized coefficient after functionalization, λ represents N loc With N cls The balance weight between .

5. The anti-collision warning method for a crane according to claim 4, characterized in that: The positioning loss function L loc , the calculation formula is as follows: L loc =αL reg +βL CIoU Where L reg represents the position regression loss function, L CIoU is the generalized intersection-over-union loss function, and α and β are hyperparameters used to adjust the functions of each part; The position regression loss function L reg The calculation formula is as follows: Among them, B is the predicted box, B′ is the target box, IoU represents the ratio of the intersection and union of the target box and the real box, and B C represents the smallest enclosed box in the positional relationship between the prediction box and the target box, and U represents the union of the prediction box and the target box; The generalized intersection-over-union loss function L CIoU The calculation formula is as follows: Among them, a is the weight parameter; v is used to measure the consistency of aspect ratio, w′ and h′ are the width and height of the target box, respectively, and w and h are the width and height of the prediction box, respectively; L cls Represents the similarity measurement loss function, and the calculation formula is as follows: Among them, x represents the true sample label, and x′ represents the detection output after the activation function.

6. The anti-collision warning method for a crane according to claim 1, characterized in that: The pre-processed historical video surveillance data is imported into a preset lightweight network structure model for training to obtain a target detection model in the crane surrounding environment, including: Dividing the historical video surveillance data into a training set, a validation set and a test set; Perform data enhancement processing on the training set to obtain an enhanced training set; The enhanced training set and the verification set are imported into a preset lightweight network structure model for training, so as to obtain a target detection model in the surrounding environment of the crane.

7. The anti-collision warning method for a crane according to claim 1, characterized in that: The determining whether the real-time distance is less than a preset threshold, and if so, performing corresponding early warning processing on the crane, includes: When the real-time distance is less than or equal to the first set distance, a voice warning is issued; When the real-time distance is less than or equal to the second set distance, the crane is controlled to stop running.

8. An anti-collision warning system for a crane, characterized in that: include: An acquisition module is used to acquire historical video surveillance data in the surrounding environment of the crane, and pre-process the historical video surveillance data to obtain the pre-processed historical video surveillance data; A training module, used for importing the pre-processed historical video surveillance data into a preset lightweight network structure model for training, so as to obtain a target detection model in the surrounding environment of the crane; The recognition and detection module is used to obtain real-time video monitoring data in the surrounding environment of the crane, and import the real-time monitoring data into the target detection model for recognition and detection to obtain target information in the surrounding environment of the crane, wherein the target information includes: category information and location information of the target; A distance measurement module is used to import the target information into a preset binocular vision distance measurement algorithm model to obtain the real-time distance between the target object and the crane; The early warning module is used to determine whether the real-time distance is less than a preset threshold value, and if so, to perform corresponding early warning processing on the crane.

9. A computer electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the anti-collision warning method for a crane according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the anti-collision warning method for a crane according to any one of claims 1 to 7 are implemented.

Images