Punching aluminum veneer processing safety early warning method and system

Abstract

The present application relates to the field of safety warning, especially to a punching aluminum veneer processing safety warning method and system. The method comprises the following steps: obtaining a punching aluminum veneer processing video, the punching aluminum veneer processing video contains a plurality of frames of punching aluminum veneer processing images; obtaining a pre-constructed safety monitoring network, and training the safety monitoring network using the punching aluminum veneer processing images to realize punching aluminum veneer processing safety warning; in the training process, the two consecutive frames of punching aluminum veneer processing images are input into the safety monitoring network in turn, the feature maps corresponding to the two frames of punching aluminum veneer processing images are obtained in the last convolution layer of the safety monitoring network, and a loss function for controlling job relationship learning is constructed; the training of the safety monitoring network is supervised by using the loss function for controlling job relationship learning. More accurate safety warning is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of safety warning, in particular to a punched aluminum veneer processing safety warning method and system. BACKGROUND

[0002] Punched aluminum veneer as a new type of building decoration material, because of its light weight, beautiful, weather resistance and other advantages, is applied to building curtain wall, indoor decoration and other fields. However, in the process of punched aluminum veneer processing, there are many safety hazards, for example, due to the fast running speed and large power of punch press, plate shearing machine and other equipment, improper operation can easily cause mechanical injury such as pinch injury and crush injury, so in order to improve the operation safety of punched aluminum veneer processing, the punched aluminum veneer processing operation process needs to be monitored, so as to issue a warning when a safety accident may occur.

[0003] At present, neural network is often used for operation safety monitoring, but neural network can only learn the associated information in a single image, but cannot analyze the associated information between images. Punched aluminum veneer processing is real-time variable, although there is no safety accident at present, but there may be a safety accident in the future, in order to predict the future safety situation, the real-time variable situation of the operation needs to be mastered. Therefore, the neural network which cannot learn the associated information between images cannot accurately realize the safety monitoring of punched aluminum veneer. How to make the neural network learn the operation variable information between images and improve the accuracy of punched aluminum veneer processing safety monitoring has become the research focus of the present application.

[0004] The patent application file with publication number CN112817286A discloses a safety monitoring system applied to automobile part processing. The method in the patent application file mainly introduces the modules and the role of each module, and does not involve neural network and other related content, so the method in the patent application file cannot solve the technical problems of the present application. SUMMARY

[0005] In order to solve the problem of how to make the neural network learn the operation variable information between images and improve the accuracy of punched aluminum veneer processing safety monitoring, the present application provides a punched aluminum veneer processing safety warning method and system.

[0006] In the first aspect, the present application provides a punched aluminum veneer processing safety warning method, which adopts the following technical scheme:

[0007] A punched aluminum veneer processing safety warning method, comprising the steps of:

[0008] Obtaining a punched aluminum veneer processing video, the punched aluminum veneer processing video contains a plurality of frames of punched aluminum veneer processing images;

[0009] The pre-constructed safety monitoring network is acquired, and the safety monitoring network is trained by using a punched aluminum veneer processing image to realize safety early warning of the punched aluminum veneer processing.

[0010] In the training process, the two continuous punched aluminum veneer processing images are sequentially input into the safety monitoring network, the feature maps corresponding to the two punched aluminum veneer processing images are acquired in the last convolutional layer of the safety monitoring network, the correlation graph formed by the feature maps of the two punched aluminum veneer processing images and the sequence formed by the job relationship change descriptors of all corresponding key point pairs in the two punched aluminum veneer processing images are acquired, and a loss function for learning the job relationship control is constructed, the loss function for learning the job relationship control is negatively correlated with the relevance of the sequence obtained by flattening the correlation graph and the sequence formed by the job relationship change descriptors.

[0011] The training of the safety monitoring network is supervised by using the loss function for learning the job relationship control.

[0012] The safety monitoring network can learn the job relationship change between the machine and the operator by designing the loss function for learning the job relationship control, so that the safety monitoring network has stronger ability to predict accidents and improves the accuracy of the processing safety early warning.

[0013] Preferably, the safety monitoring network is acquired by constructing a safety monitoring network.

[0014] The convolutional neural network is constructed, and the convolutional neural network is used as the safety monitoring network.

[0015] Preferably, the method for constructing the loss function for learning the job relationship control comprises the following steps.

[0016] ;

[0017] Wherein, The sequence formed by the job relationship change descriptors of all corresponding key point pairs in the two punched aluminum veneer processing images, The sequence obtained by flattening the correlation graph, The relevance of the sequence and the sequence , and S represents the loss function for learning the job relationship control.

[0018] Preferably, the job relationship change descriptors of the corresponding key point pairs in the two punched aluminum veneer processing images are acquired by the following steps.

[0019] Segmenting each frame of the punched aluminum veneer processing image to obtain an operator region and a machine region;

[0020] The key points in the operator region of one frame of the punched aluminum veneer processing image and the nearest key points in the machine region form a key point pair, the vectors formed by the two key points in any key point pair are obtained, the vectors formed by the corresponding key point pairs in another frame of the punched aluminum veneer processing image are obtained, the difference vector of the vectors obtained by the two corresponding key point pairs in the two frames of the punched aluminum veneer processing image is denoted as a work relationship change description vector based on the key point pair; the work relationship change description vector is multiplied by the module length to obtain a work relationship change description sub of the key point pair.

[0021] The present application considers that the relative positions of different operators and machines are different, and thus the work relationship between each part of the operator and each part of the machine region is accurately reflected by the relative position vectors of the key points in the operator region and the key points in the machine region, thereby providing a basis for subsequent construction of the work relationship change description sub.

[0022] Preferably, the segmenting each frame of the punched aluminum veneer processing image to obtain an operator region and a machine region comprises:

[0023] The punched aluminum veneer processing image is input into a pre-trained image segmentation network to obtain the operator region and the machine region in each frame of the punched aluminum veneer processing image.

[0024] Preferably, the obtaining the association graph formed by the feature maps of the two frames of the punched aluminum veneer processing image comprises:

[0025] The feature maps obtained from each frame of the punched aluminum veneer processing image are superimposed to obtain a comprehensive feature map of each frame of the punched aluminum veneer processing image.

[0026] The difference image between the comprehensive feature map of one frame of the punched aluminum veneer processing image and the comprehensive feature map of another frame of the punched aluminum veneer processing image is taken as the association graph.

[0027] The present application obtains the association graph by the difference between the feature maps, which is simple to implement and has high implementation efficiency.

[0028] Preferably, the punched aluminum veneer processing safety early warning is realized, comprising:

[0029] The newly collected punched aluminum veneer processing video is input into the trained safety monitoring network to obtain a safety detection result, and if the safety detection result indicates that there is a safety hazard, an early warning is issued.

[0030] The safety monitoring network of the present application can predict the possible situation of a safety accident as early as possible, thereby giving an early warning and reminding, and effectively reducing the probability of a safety accident.

[0031] Preferably, the association graph is inputted to the next stage of the safety monitoring network together with the feature Figure One The next stage of the safety monitoring network.

[0032] The present application inputs the association graph to the next stage of the safety monitoring network, so that the safety monitoring network can learn the change information between images, and the safety monitoring network can learn the change information of the working relationship time sequence, thereby providing a basis for accurate safety monitoring.

[0033] In a second aspect, the present application provides a punched aluminum veneer processing safety early warning system, which adopts the following technical scheme:

[0034] A punched aluminum veneer processing safety early warning system, comprising: a processor and a memory, the memory stores computer program instructions, when the computer program instructions are executed by the processor, the above-mentioned punched aluminum veneer processing safety early warning method is realized.

[0035] By adopting the above technical scheme, the above-mentioned punched aluminum veneer processing safety early warning method is generated into a computer program and stored in the memory, so as to be loaded and executed by the processor, so as to manufacture a terminal device according to the memory and the processor, and facilitate use.

[0036] The present application has the following technical effects:

[0037] The present application supervises the safety monitoring network to learn the change of the working relationship between the machine and the operator by designing a loss function for controlling the learning of the working relationship, so that the safety monitoring network has stronger ability to predict the occurrence of accidents, and the accuracy of the processing safety early warning is improved.

[0038] Further, when designing the loss function, the association graph obtained by constraining the feature map has a correlation with the working relationship change description, so that the association graph can have information reflecting the working relationship change characteristics, so that the safety monitoring network can learn the working relationship change information. BRIEF DESCRIPTION OF DRAWINGS

[0039] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description read in conjunction with the accompanying drawings. In the drawings, several embodiments of the present application are illustrated by way of example and not limitation. Identical or corresponding elements are provided with identical or corresponding reference numerals.

[0040] Figure 1 is a flowchart of a method in a punched aluminum veneer processing safety early warning method according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0042] It should be understood that when the claims, the specification, and the drawings of the present application use the terms "first", "second", etc., they are only used to distinguish different objects, and are not used to describe a specific sequence. The terms "include" and "contain" used in the specification and claims of the present application indicate the presence of the described features, whole, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, whole, steps, operations, elements, components and / or sets thereof.

[0043] The embodiment of the present application discloses a punching aluminum veneer processing safety early warning method, referring to Figure 1 , comprising steps S1-S2:

[0044] S1: obtaining a punching aluminum veneer processing video, wherein the punching aluminum veneer processing video contains a plurality of frames of punching aluminum veneer processing images.

[0045] Specifically, a camera is installed directly above the punching aluminum veneer processing machine, and the camera is used to collect the video during the punching aluminum veneer processing process, which is recorded as the punching aluminum veneer processing video. Each frame of image in the punching aluminum veneer processing video is recorded as a punching aluminum veneer processing image.

[0046] S2: obtaining a pre-constructed safety monitoring network, and using the punching aluminum veneer processing image to complete the training of the safety monitoring network to realize the punching aluminum veneer processing safety early warning.

[0047] S20: obtaining a pre-constructed safety monitoring network.

[0048] Preferably, as an example, the pre-constructed safety monitoring network is obtained, comprising:

[0049] A convolutional neural network is constructed, and the network structure in the AlexNet algorithm is adopted in the embodiment, and other embodiments can adopt other networks, and the embodiment does not make specific limitation. The constructed convolutional neural network is used as the safety monitoring network.

[0050] S21: using the punching aluminum veneer processing image to complete the training of the safety monitoring network.

[0051] It should be noted that in order to enable the safety monitoring network to learn the job variation information between images, a loss function needs to be designed to control the training process of the safety monitoring network.

[0052] S210: Constructing a data set.

[0053] Preferably, as an example, the data set is constructed, including:

[0054] Two adjacent frames of punched aluminum veneer processing images are taken as a group, and it is artificially judged whether there is a safety accident in the punched aluminum veneer processing images in any group. If there is a safety accident, a preset length of time period before the corresponding time of the punched aluminum veneer processing images in the group is taken as a research period. If the corresponding time of the punched aluminum veneer images in a group belongs to the research period, the label of the group is set to [1, 0]. If the corresponding time of the punched aluminum veneer images in a group does not belong to the research period, the label of the group is set to [0, 1].

[0055] It can be understood that a series of safety hazard operation interaction behaviors will generally exist before a safety accident, so by analyzing the behavior changes in the research period before the safety accident, better safety accident prediction can be achieved. Therefore, if the operator makes some behaviors similar to those in the research period, it means that a safety accident is likely to occur, so an early warning signal is given in time to stop the relevant behavior. By setting different labels for each group belonging to the research period and each group not belonging to the research period, the behaviors causing safety accidents and the behaviors not causing safety accidents can be distinguished.

[0056] S211: Constructing a loss function for controlling operation relationship learning.

[0057] It should be noted that in order to more accurately predict the occurrence of safety accidents, the safety monitoring network needs to learn the change information of the operation interaction relationship between the machine and the operator, so that the safety monitoring network can predict the future occurrence of safety accidents according to the change of the operation interaction relationship between the machine and the operator.

[0058] Preferably, as an example, the loss function for controlling operation relationship learning is constructed, including:

[0059] In the training process, two frames of punched aluminum veneer images in a group are sequentially input into the safety monitoring network, and the feature maps corresponding to the two frames of punched aluminum veneer processing images are obtained in the last convolutional layer of the safety monitoring network.

[0060] The loss function for controlling operation relationship learning satisfies the relationship:

[0061]

[0062] Wherein, the operation relationship change descriptor composed of the corresponding key point pairs in the two frames of punched aluminum veneer processing images is obtained, and the association graph composed of the feature maps of the two frames of punched aluminum veneer processing images is obtained. a sequence composed of operation relationship change descriptors of all corresponding key point pairs in two frames of punched aluminum veneer processing images, a sequence obtained by flattening the association graph, a sequence and a sequence correlation, and S represents a loss function for controlling operation relationship learning.

[0063] It can be understood that, in order to enable the safety monitoring network to learn the operation relationship change information, the association information between the feature maps extracted by the safety monitoring network needs to be able to reflect the operation relationship change information. The operation relationship change descriptor reflects the operation relationship change information between images. The association graph reflects the association information between the feature maps of different images. In order to enable the association graph to contain the operation relationship change information, the data in the association graph should have a large correlation with the data of the operation relationship change descriptor.

[0064] The above embodiments relate to operation relationship change descriptors and association graphs. The following describes methods for determining operation relationship change descriptors and association graphs.

[0065] First, an operation relationship change descriptor composed of corresponding key point pairs in two frames of punched aluminum veneer processing images is obtained.

[0066] It should be noted that different operation actions correspond to different relative positions between the parts of the operator and the parts of the machine. Therefore, the operation interaction relationship between the operator and the machine can be reflected by the relative relationship between the positions of the machine and the positions of the operator.

[0067] Preferably, as an example, the operation relationship change descriptor composed of corresponding key point pairs in two frames of punched aluminum veneer processing images includes:

[0068] segmenting each frame of punched aluminum veneer processing image to obtain an operator region and a machine region;

[0069] forming a key point pair from a key point in the operator region and a nearest key point in the machine region of one frame of punched aluminum veneer processing image, obtaining a vector composed of two key points in any key point pair, obtaining a vector composed of corresponding key point pairs in another frame of punched aluminum veneer processing image, and taking the difference vector of the vectors obtained from two corresponding key point pairs in two frames of punched aluminum veneer processing images as an operation relationship change description vector based on the key point pair; and multiplying the included angle between the operation relationship change description vector and the horizontal right unit vector by the module length to obtain the operation relationship change descriptor of the key point pair.

[0070] It can be understood that the vector composed of the key points of the one frame of punched aluminum veneer processing image reflects the relative position information of one part of the machine and one part of the worker, and through the information, the interaction between the worker and the machine can be reflected. The difference vector of the vectors obtained by the corresponding key points in the two frames of punched aluminum veneer processing images reflects the interaction change information of the machine and the worker.

[0071] It should be added that the worker area and the machine area obtained by segmenting each frame of punched aluminum veneer processing image include:

[0072] The image segmentation network is constructed, and in this embodiment, VGG16 is used as the image segmentation network, and other embodiments can use other networks, and this embodiment does not make specific limitations.

[0073] The punched aluminum veneer image with labels is used as a training sample, and the training of the image segmentation network is completed using the training sample.

[0074] Each frame of punched aluminum veneer processing image is input into the pre-trained image segmentation network to obtain the worker area and the machine area in each frame of punched aluminum veneer processing image.

[0075] Then, the correlation graph composed of the feature maps of the two frames of punched aluminum veneer processing images is obtained.

[0076] Preferably, as an example, the correlation graph composed of the feature maps of the two frames of punched aluminum veneer processing images includes:

[0077] The feature maps obtained from each frame of punched aluminum veneer processing image are superimposed to obtain the comprehensive feature map of each frame of punched aluminum veneer processing image.

[0078] The difference image between the comprehensive feature map of one frame of punched aluminum veneer processing image in each group and the comprehensive feature map of the other frame of punched aluminum veneer processing image is used as the correlation graph.

[0079] It can be understood that the comprehensive feature map reflects the comprehensive feature information of the punched aluminum veneer processing image extracted by the safety monitoring network, and the difference image of the comprehensive feature maps of the two frames of punched aluminum veneer images reflects the change information of the information in the two frames of punched aluminum veneer images. Since the punched aluminum veneer image mainly reflects the worker, the machine and the interaction information between the worker and the machine, the inherent information of the worker and the machine will not change, only the work information of the worker and the machine will change, so the difference between the two frames of punched aluminum veneer images is mainly the difference of the work and the work interaction information.

[0080] S212: Supervise the training of the safety monitoring network using the loss function for controlling the work relationship learning.

[0081] Preferably, as an example, the training of the safety monitoring network is supervised by a loss function of control task relation learning, including:

[0082] The loss value is calculated by the loss function of control task relation learning, and is recorded as a first loss value.

[0083] The features corresponding to the correlation graph and the two frames of punched aluminum veneer image pairs are Figure One And flow into the next link of the safety monitoring network to participate in the operation of the safety monitoring network, and finally obtain an output result in the safety monitoring network, based on the output result and the label, the loss value is calculated by using the loss function of the safety monitoring network, and is recorded as a second loss value.

[0084] The first loss value and the first loss value are accumulated to obtain a comprehensive loss value, and based on the comprehensive loss value, the parameters in the safety monitoring network are updated by using the gradient descent method.

[0085] It should be noted that the network parameters are updated by using the gradient descent method based on the loss value, which is a prior art, and will not be described here.

[0086] S22: realize the safety pre-warning of the punched aluminum veneer processing.

[0087] Preferably, as an example, to realize the safety pre-warning of the punched aluminum veneer processing, including:

[0088] The newly collected punched aluminum veneer processing video is input into the safety monitoring network after training to obtain a safety detection result, and if the safety detection result exists safety hidden danger, an early warning is issued.

[0089] The embodiment of the application also discloses a punched aluminum veneer processing safety pre-warning system, including a processor and a memory, the memory stores computer program instructions, when the computer program instructions are executed by the processor, a punched aluminum veneer processing safety pre-warning method according to the application is realized.

[0090] The above system also includes a communication bus and a communication interface and other components familiar to those skilled in the art, the setting and function of which are known in the art, therefore, will not be described here.

[0091] In this disclosure, a "storage medium" can be any available medium that can be accessed by a general purpose or special purpose computer system, apparatus, or device to store, retrieve, or store and retrieve information. A storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, holographic, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a storage medium would be a portable magnetic disk, a portable optical disk, random access memory (RAM), a cache, a dynamic random access memory (DRAM), a floppy disk attached to a disk drive, a hard disk drive, a solid state drive, a magnetic tape drive, a holographic storage medium, a direct access storage device, a flash memory device, a network storage device, or a suitable combination of the foregoing. A storage medium can reside in a computer readable storage medium holder, which can be a device or location in a storage medium, a medium, or a component of either, for example a cache or RAM located in a server or a memory bank. A storage medium can be associated with one or more computer readable storage media, devices, or storage media holders.

[0092] While the present disclosure has shown and described several embodiments of an application, it is to be understood that the above-described embodiments are merely representative of many possible embodiments which can be made and that many changes in detail of structure and arrangement of parts can be made without departing from the spirit and scope of this application. Accordingly, other embodiments are within the scope of the following claims:

[0093] The above are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application, therefore: any equivalent changes made on the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A safety early warning method for perforated aluminum single-panel processing, characterized in that, Including the following steps: Obtain a video of perforated aluminum panel processing, which contains several frames of perforated aluminum panel processing images. A pre-constructed safety monitoring network is acquired, and images of perforated aluminum panels are used to train the safety monitoring network in order to achieve safety early warning for perforated aluminum panel processing. During training, two consecutive frames of perforated aluminum panel processing images are sequentially input into the safety monitoring network. In the last convolutional layer of the safety monitoring network, the feature maps corresponding to the two frames of perforated aluminum panel processing images are obtained. The association map formed by the feature maps of the two frames of perforated aluminum panel processing images and the sequence formed by the operation relationship change descriptors formed by all corresponding key point pairs in the two frames of perforated aluminum panel processing images are obtained. A loss function controlling the learning of operation relationships is constructed. The correlation between the loss function controlling the learning of operation relationships and the sequence formed by the flattened association map and the sequence formed by the operation relationship change descriptors is negatively correlated. A method for obtaining the operation relationship variation descriptor formed by corresponding key point pairs in two frames of perforated aluminum panel processing images, including: Each frame of the perforated aluminum panel processing image is segmented to obtain the operator area and the machine area; In a single frame of a perforated aluminum panel processing image, key points in the worker area and the nearest key point in the machine area are paired to form key point pairs. The vector formed by the two key points in any key point pair is obtained. The vector formed by the corresponding key point pair in another frame of the perforated aluminum panel processing image is then obtained. The difference vector between the vectors obtained from the two corresponding key point pairs in the two frames of the perforated aluminum panel processing image is denoted as the operation relationship change description vector based on the key point pair. The angle between the operation relationship change description vector and the horizontal rightward unit vector is multiplied by the modulus to obtain the operation relationship change descriptor for the key point pair. The vector formed by the key point pairs in a single frame of the perforated aluminum panel processing image reflects the relative position information of a part of the machine and a part of the worker, reflecting the interaction between the worker and the machine. The difference vector between the vectors formed by the corresponding key point pairs in the two frames of the perforated aluminum panel processing image reflects the interaction change information between the machine and the worker. A correlation graph composed of feature maps from two frames of perforated aluminum panel processing images is obtained, including: The feature maps obtained from each frame of perforated aluminum panel processing image are superimposed to obtain the comprehensive feature map of each frame of perforated aluminum panel processing image. The difference image between the comprehensive feature map of one frame of perforated aluminum panel processing image and the comprehensive feature map of another frame of perforated aluminum panel processing image is used as the correlation map. The training of the safety monitoring network is supervised using a loss function learned from control job relationships. This includes: calculating a loss value using the loss function learned from control job relationships, denoted as the first loss value; feeding the association graph and the feature maps corresponding to the two frames of perforated aluminum single-panel images into the next stage of the safety monitoring network for computation, and obtaining the output result at the end of the safety monitoring network; calculating a loss value using the loss function built into the safety monitoring network based on the output result and the label, denoted as the second loss value; summing the first loss value and the second loss value to obtain a comprehensive loss value; and updating the parameters in the safety monitoring network using gradient descent based on the comprehensive loss value.

2. The safety early warning method for perforated aluminum single-panel processing according to claim 1, characterized in that, The acquisition of the pre-built security monitoring network includes: Convolutional neural networks are constructed and used as security monitoring networks.

3. The safety early warning method for perforated aluminum single-panel processing according to claim 1, characterized in that, The segmentation processing of each frame of perforated aluminum single-panel processing image to obtain the operator area and machine area includes: Each frame of the perforated aluminum panel processing image is input into a pre-trained image segmentation network to obtain the operator area and machine area in each frame of the perforated aluminum panel processing image.

4. The safety early warning method for perforated aluminum single-panel processing according to claim 1, characterized in that, The method for implementing safety early warning in perforated aluminum panel processing includes: The newly acquired video of the perforated aluminum panel processing is input into the safety monitoring network after training to obtain the safety detection results. If the safety detection results indicate that there are safety hazards, an early warning is issued.

5. The safety early warning method for perforated aluminum single-panel processing according to claim 1, characterized in that, The association graph, along with the feature graph, flows into the next stage of the security monitoring network.

6. A safety early warning system for perforated aluminum single-panel processing, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement a safety early warning method for perforated aluminum single-panel processing according to any one of claims 1-5.

Images