Safety early warning method and system for punching aluminum veneer machining

Abstract

The invention relates to the field of safety early warning, in particular to a safety early warning method and system for punching aluminum veneer machining. The method comprises the following steps: acquiring a punching aluminum veneer processing video, wherein the punching aluminum veneer processing video comprises a plurality of frames of punching aluminum veneer processing images; acquiring a pre-constructed safety monitoring network, and completing training of the safety monitoring network by using the punching aluminum veneer processing image to realize punching aluminum veneer processing safety early warning; in the training process, two continuous frames of punched aluminum veneer processing images are sequentially input into the safety monitoring network, feature maps corresponding to the two frames of punched aluminum veneer processing images are obtained in the last convolutional layer of the safety monitoring network, and a loss function for controlling operation relation learning is constructed; and supervising the training of the safety monitoring network by using a loss function of control operation relation learning. And more accurate safety early warning is realized.

Description

Technical Field

[0001] The present invention relates to the field of safety warning, and particularly to a safety warning method and system for the processing of punched aluminum veneers. Background Art

[0002] As a new type of building decoration material, punched aluminum veneers are applied to fields such as building curtain walls and interior decorations due to their advantages such as light weight, beauty, and strong weather resistance. However, during the processing of punched aluminum veneers, there are many potential safety hazards. For example, due to the high operating speed and large force of equipment such as punching presses and plate shears, mechanical injuries such as pinching and crushing are likely to occur if the operation is improper. Therefore, in order to improve the operation safety of punched aluminum veneer processing, it is necessary to monitor the operation process of punched aluminum veneer processing, so as to issue a warning reminder when a safety accident may occur.

[0003] Currently, neural networks are often used for operation safety monitoring. However, neural networks generally can only learn the correlation information in a single image and cannot analyze the correlation information between images. The processing operation of punched aluminum veneers is constantly changing. Although there is no safety accident currently, there may be a safety accident in the future. In order to predict the future safety situation, it is necessary to master the real-time changes in the processing operation. Therefore, it is impossible to accurately implement the safety monitoring of punched aluminum veneers through a neural network that cannot learn the correlation information between images. How to enable the neural network to learn the operation change information between images and improve the accuracy of safety monitoring of punched aluminum veneer processing has become the research focus of this solution.

[0004] The patent application document with the publication number CN112817286A discloses a safety monitoring system applied to the processing of automotive parts. The method in this patent application document mainly introduces the modules and the functions of each module, and does not involve relevant content such as neural networks. Therefore, it is impossible to solve the technical problems of this solution by using the method in this patent application document. Summary of the Invention

[0005] In order to solve the problem of how to enable the neural network to learn the operation change information between images and improve the accuracy of safety monitoring of punched aluminum veneer processing, the present invention provides a safety warning method and system for the processing of punched aluminum veneers.

[0006] In a first aspect, the present invention provides a safety warning method for the processing of punched aluminum veneers, adopting the following technical solution: A safety warning method for the processing of punched aluminum veneers includes the steps of: Obtaining a video of punched aluminum veneer processing, where the video of punched aluminum veneer processing contains a plurality of frames of punched aluminum veneer processing images; Obtaining a pre-constructed safety monitoring network, and using the punched aluminum veneer processing images to complete the training of the safety monitoring network to achieve safety warning for the processing of punched aluminum veneers; During the training process, two consecutive frames of the punched aluminum veneer processing images are sequentially input into the safety monitoring network. Feature maps corresponding to the two frames of the punched aluminum veneer processing images are obtained in the last convolutional layer of the safety monitoring network. An association graph composed of the feature maps of the two frames of the punched aluminum veneer processing images and a sequence composed of job relationship change descriptors formed by all corresponding key point pairs in the two frames of the punched aluminum veneer processing images are obtained. A loss function for controlling job relationship learning is constructed, and the loss function for controlling job relationship learning is negatively correlated with the correlation between the sequence obtained by flattening the association graph and the sequence composed of job relationship change descriptors. Use the loss function for controlling job relationship learning to supervise the training of the safety monitoring network.

[0007] In the present invention, by designing a loss function for controlling job relationship learning to supervise the safety monitoring network to learn the changes in the job relationship between the machine and the operator, the ability of the safety monitoring network to predict accidents is stronger, and the accuracy of the processing safety warning is improved. Further, when designing the loss function, by constraining the correlation between the association graph obtained from the feature maps and the job relationship change descriptors, information reflecting the job relationship change characteristics can be included in the association graph, so that the safety monitoring network can learn the job relationship change information.

[0008] Preferably, the obtaining of the pre-constructed safety monitoring network includes: Construct a convolutional neural network and use the convolutional neural network as the safety monitoring network.

[0009] Preferably, the construction method of the loss function for controlling job relationship learning includes: ; wherein, represents the sequence composed of job relationship change descriptors formed by all corresponding key point pairs in the two frames of the punched aluminum veneer processing images, represents the sequence obtained by flattening the association graph, represents the sequence and the sequence correlation, and S represents the loss function for controlling job relationship learning.

[0010] Preferably, obtaining the job relationship change descriptors formed by the corresponding key point pairs in the two frames of the punched aluminum veneer processing images includes: Segment each frame of the punched aluminum veneer processing image to obtain the operator area and the machine area; For a key point in the operator area of a processed image of a punched aluminum single panel and the nearest key point in the machine area, form a key point pair. Obtain the vector formed by the two key points in any key point pair, and obtain the vector formed by the corresponding key point pair in another processed image of a punched aluminum single panel. Denote the difference vector between the vectors obtained from the two corresponding key point pairs in the two processed images of the punched aluminum single panel as the operation relationship change description vector based on this key point pair; Multiply the included angle between the operation relationship change description vector and the unit vector horizontally to the right by the modulus length to obtain the operation relationship change descriptor for this key point pair.

[0011] In the present invention, considering that the relative positions of different operators and machines in the operation relationship are different, the relative position vector between the key points in the operator area and the key points in the machine area is used to accurately reflect the operation relationship between each part of the operator and each part of the machine area, thereby providing a basis for constructing the operation relationship change descriptor subsequently.

[0012] Preferably, the segmentation process of each processed image of a punched aluminum single panel to obtain the operator area and the machine area includes: Input each processed image of a punched aluminum single panel into a pre-trained image segmentation network to obtain the operator area and the machine area in each processed image of a punched aluminum single panel.

[0013] Preferably, the obtaining of the correlation graph composed of the feature maps of two processed images of a punched aluminum single panel includes: Perform superposition processing on the feature maps obtained from each processed image of a punched aluminum single panel to obtain the comprehensive feature map of each processed image of a punched aluminum single panel; Use the difference image between the comprehensive feature map of one processed image of a punched aluminum single panel and the comprehensive feature map of another processed image of a punched aluminum single panel as the correlation graph.

[0014] The present invention obtains the correlation graph by subtracting the feature maps, and this method is simple to implement and has high implementation efficiency.

[0015] Preferably, the realization of safety warning for the processing of punched aluminum single panels includes: Input the newly collected video of the processing of punched aluminum single panels into the trained safety monitoring network to obtain the safety detection result. If the safety detection result is that there is a potential safety hazard, then issue a warning.

[0016] The present invention can predict the possible situations of safety accidents early through the safety monitoring network, thereby giving an early warning reminder and effectively reducing the probability of safety accidents.

[0017] Preferably, the correlation graph together with the features Figure 1 flows into the next link of the safety monitoring network.

[0018] The present invention also inputs the association graph into the next link of the safety monitoring network, enabling the safety monitoring network to learn the change information between images and the sequential change information of the operation relationship, providing a basis for accurate safety monitoring.

[0019] In a second aspect, the present invention provides a safety warning system for the processing of punched aluminum single plates, adopting the following technical solutions: A safety warning system for the processing of punched aluminum single plates includes: a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned safety warning method for the processing of punched aluminum single plates is implemented.

[0020] By adopting the above technical solutions, the above-mentioned safety warning method for the processing of punched aluminum single plates is generated into a computer program and stored in the memory to be loaded and executed by the processor, thereby manufacturing a terminal device according to the memory and the processor for convenient use.

[0021] The present invention has the following technical effects: By designing a loss function for controlling the learning of the operation relationship, the present invention supervises the safety monitoring network to learn the change situation of the operation relationship between the machine and the operator, thereby making the safety monitoring network have a stronger ability to predict accidents and improving the accuracy of the processing safety warning. Furthermore, when designing the loss function, by constraining the association graph obtained from the feature map to have a correlation with the operation relationship change descriptor, information reflecting the operation relationship change characteristics can be included in the association graph, enabling the safety monitoring network to learn the operation relationship change information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By referring to the drawings and reading the following detailed description, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become easily understandable. In the drawings, several embodiments of the present invention are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0023] Figure 1 is a flowchart of the method in a safety warning method for the processing of punched aluminum single plates according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] It should be understood that when terms such as "first" and "second" are used in the claims, the specification and the drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0026] An embodiment of the present invention discloses a safety warning method for punching aluminum veneer processing. Referring to Figure 1 , it includes steps S1 - step S2: S1: Obtain the punching aluminum veneer processing video, and the punching aluminum veneer processing video contains several frames of punching aluminum veneer processing images.

[0027] Specifically, install a camera directly above the punching aluminum veneer processing machine, and use the camera to collect the video during the punching aluminum veneer processing, which is recorded as the punching aluminum veneer processing video. Each frame of the image in the punching aluminum veneer processing video is recorded as the punching aluminum veneer processing image.

[0028] S2: Obtain the pre - constructed safety monitoring network, and use the punching aluminum veneer processing images to complete the training of the safety monitoring network to achieve safety warning for punching aluminum veneer processing.

[0029] S20: Obtain the pre - constructed safety monitoring network.

[0030] Preferably, as an example, obtaining the pre - constructed safety monitoring network includes: Construct a convolutional neural network. In this embodiment, the network structure in the AlexNet algorithm is adopted, and other embodiments can adopt other networks, which are not specifically limited in this embodiment. The constructed convolutional neural network is used as the safety monitoring network.

[0031] S21: Use the punching aluminum veneer processing images to complete the training of the safety monitoring network.

[0032] It should be noted that in order to enable the safety monitoring network to learn the operation change information between images, a loss function needs to be designed to control the training process of the safety monitoring network.

[0033] S210: Construct a data set.

[0034] Preferably, as an example, constructing a data set includes: Take every two adjacent processed images of punched aluminum single plates as a group, and manually judge whether there is a safety accident in the processed images of punched aluminum single plates in any group. If there is a safety accident, take the preset length of time period before the corresponding time of the processed images of punched aluminum single plates in this group as the research time period; if the corresponding time of the punched aluminum single plate images in a group belongs to the research time period, set the label of this group to [1, 0]. If the corresponding time of the punched aluminum single plate images in a group does not belong to the research time period, set the label of this group to [0, 1].

[0035] It can be understood that generally there will be a series of operation interaction behaviors with potential safety hazards before a safety accident. Therefore, by analyzing the changes in behaviors during the research time period before the occurrence of a safety accident, better prediction of safety accidents can be achieved. Thus, if an operator makes some behaviors similar to those during the research time period, it indicates that a safety accident is very likely to occur, so a warning signal is given in a timely manner to stop the relevant behaviors. By setting different labels for the groups belonging to the research time period and the groups not belonging to the research time period, the behaviors that cause safety accidents and the behaviors that do not cause safety accidents can be distinguished.

[0036] S211: Construct a loss function for learning the control operation relationship.

[0037] It should be noted that in order to more accurately predict the occurrence of safety accidents, the safety monitoring network needs to learn the change information of the operation interaction relationship between the machine and the operator, and then the safety monitoring network can predict the future occurrence of safety accidents based on the change situation of the operation interaction relationship between the machine and the operator.

[0038] Preferably, as an example, constructing a loss function for learning the control operation relationship includes: During the training process, input two processed images of punched aluminum single plates in a group into the safety monitoring network in sequence, and obtain the feature maps corresponding to the two processed images of punched aluminum single plates respectively in the last convolutional layer of the safety monitoring network.

[0039] The loss function for learning the control operation relationship satisfies the relational expression:

[0040] Among them, obtain the operation relationship change descriptor composed of corresponding key point pairs in the two processed images of punched aluminum single plates, and obtain the correlation graph composed of the feature maps of the two processed images of punched aluminum single plates, represents the sequence composed of operation relationship change descriptors of all corresponding key point pairs in the two processed images of punched aluminum single plates, represents the sequence obtained by flattening the correlation graph, represents the sequence and the sequence The relevance, where S represents the loss function for controlling job relationship learning.

[0041] It can be understood that in order for the safety monitoring network to learn the job relationship change information, it is necessary to ensure that the correlation information between the feature maps extracted by the safety monitoring network can reflect the job relationship change information. The job relationship change descriptor reflects the job relationship change information between images, and the correlation graph reflects the correlation information between the feature maps of different images. To make the correlation graph contain job relationship change information, the data in the correlation graph should have a relatively large correlation with the data of the job relationship change descriptor.

[0042] The above embodiments involve the job relationship change descriptor and the correlation graph. Next, the methods for determining the job relationship change descriptor and the correlation graph will be described.

[0043] First, obtain the job relationship change descriptor composed of corresponding key point pairs in two frames of punched aluminum single-board processing images.

[0044] It should be noted that since the relative positions of the various parts of the operator and the machine are different for different job actions, the job interaction relationship between the operator and the machine can be reflected by the relative relationship between the various positions of the machine and the various positions of the operator.

[0045] Preferably, as an example, obtaining the job relationship change descriptor composed of corresponding key point pairs in two frames of punched aluminum single-board processing images includes: Performing segmentation processing on each frame of punched aluminum single-board processing image to obtain the operator area and the machine area; Forming a key point pair by the key point in the operator area of one frame of punched aluminum single-board processing image and the nearest key point in the machine area, obtaining the vector formed by the two key points in any key point pair, obtaining the vector formed by the corresponding key point pair in another frame of punched aluminum single-board processing image, and denoting the difference vector of the vectors obtained from the two corresponding key point pairs in the two frames of punched aluminum single-board processing images as the job relationship change description vector based on this key point pair; multiplying the angle between the job relationship change description vector and the unit vector horizontally to the right by the modulus length to obtain the job relationship change descriptor for this key point pair.

[0046] It can be understood that the vector formed by the key point pair in one frame of punched aluminum single-board processing image reflects the relative position information between a part of the machine and a part of the operator, and the interaction situation between the operator and the machine can be reflected through this information. The difference vector of the vectors obtained from the corresponding key points in the two frames of punched aluminum single-board processing images reflects the interaction change information between the machine and the operator.

[0047] It should be added that performing segmentation processing on each frame of punched aluminum single-board processing image to obtain the operator area and the machine area includes: An image segmentation network is constructed. This embodiment uses VGG16 as the image segmentation network. Other embodiments may use other networks, and this embodiment does not make any specific restrictions.

[0048] The perforated aluminum veneer images with labels are used as training samples, and the training samples are used to complete the training of the image segmentation network.

[0049] Each frame of punching aluminum veneer processing image is input into a pre-trained image segmentation network to obtain the operator area and machine area in each frame of punching aluminum veneer processing image.

[0050] Then, a correlation graph consisting of feature graphs of two frames of punched aluminum veneer processing images is obtained.

[0051] Preferably, as an example, obtaining a correlation graph consisting of feature graphs of two frames of punching aluminum veneer processing images includes: The feature maps obtained from each frame of the punching aluminum veneer processing image are superimposed to obtain a comprehensive feature map of each frame of the punching aluminum veneer processing image; The difference image between the comprehensive feature map of one frame of punching aluminum veneer processing image and the comprehensive feature map of another frame of punching aluminum veneer processing image in each group is used as the correlation map.

[0052] It can be understood that the comprehensive feature map reflects the comprehensive feature information of the perforated aluminum veneer processing image extracted by the safety monitoring network, and the difference image of the comprehensive feature map of the two frames of perforated aluminum veneer images reflects the change information of the information in the two frames of perforated aluminum veneer images. Since the perforated aluminum veneer images mainly reflect the operator, the machine and the interaction information between the operator and the machine, the inherent information of the operator and the machine will not change, and only the operation information of the operator and the machine will change. Therefore, the difference between the two frames of perforated aluminum veneer images is mainly the difference in the operation and the operation interaction information.

[0053] S212: Supervise the training of the safety monitoring network using the loss function learned from the control operation relationship.

[0054] Preferably, as an example, the training of the safety monitoring network is supervised by the loss function of the control operation relationship learning, including: The loss value is calculated using the loss function learned from the control operation relationship and is recorded as the first loss value.

[0055] The features corresponding to the correlation graph and the two frames of perforated aluminum veneer images are Figure 1 And it flows into the next link of the security monitoring network to participate in the security monitoring network operation, and obtains the output result at the end of the security monitoring network. Based on the output result and the label, the loss value is calculated using the loss function of the security monitoring network and recorded as the second loss value.

[0056] Accumulate the first loss value and the first loss value to obtain a comprehensive loss value, and based on the comprehensive loss value, use the gradient descent method to update the parameters in the safety monitoring network.

[0057] It should be noted that updating the network parameters using the gradient descent method based on the loss value is a prior art, and will not be elaborated here.

[0058] S22: To achieve safety warning for the processing of punched aluminum single plates.

[0059] Preferably, as an example, to achieve safety warning for the processing of punched aluminum single plates, including: Input the newly collected video of the processing of punched aluminum single plates into the trained safety monitoring network to obtain a safety detection result. If the safety detection result indicates a potential safety hazard, a warning is issued.

[0060] An embodiment of the present invention also discloses a safety warning system for the processing of punched aluminum single plates, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a safety warning method for the processing of punched aluminum single plates according to the present invention is implemented.

[0061] The above system also includes other components well-known to those skilled in the art such as a communication bus and a communication interface, and their settings and functions are known in the art, so they will not be elaborated here.

[0062] In the present invention, the aforementioned memory can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. For example, a computer-readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory, a dynamic random access memory, a static random access memory, an enhanced dynamic random access memory, a high-bandwidth memory, a hybrid storage cube, etc., or any other medium that can be used to store the required information and can be accessed by an application program, a module, or both. Any such computer storage medium can be a part of the device or accessible or connectable to the device.

[0063] Although this specification has shown and described multiple embodiments of the present invention, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art will think of many changes, alterations, and alternative ways without departing from the spirit and idea of the present invention. It should be understood that various alternative solutions to the embodiments of the present invention described herein can be adopted in the process of practicing the present invention.

[0064] The above are all preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A safety warning method for the processing of punched aluminum single plates, characterized in that, Including the steps of: Obtaining a processing video of punched aluminum single plates, where the processing video of punched aluminum single plates contains several frames of processed images of punched aluminum single plates; Obtaining a pre-constructed safety monitoring network, and using the processed images of punched aluminum single plates to complete the training of the safety monitoring network to achieve safety warning for the processing of punched aluminum single plates; During the training process, input two consecutive frames of processed images of punched aluminum single plates into the safety monitoring network in sequence. In the last convolutional layer of the safety monitoring network, obtain the feature maps corresponding to each of the two frames of processed images of punched aluminum single plates, obtain the association graph formed by the feature maps of the two frames of processed images of punched aluminum single plates and the sequence formed by the operation relationship change descriptors composed of all corresponding key point pairs in the two frames of processed images of punched aluminum single plates, and construct a loss function for controlling the learning of operation relationships. The loss function for controlling the learning of operation relationships is negatively correlated with the correlation between the sequence obtained by flattening the association graph and the sequence formed by the operation relationship change descriptors; Use the loss function for controlling the learning of operation relationships to supervise the training of the safety monitoring network.

2. The punching aluminum single board processing safety warning method according to claim 1, characterized in that The obtaining of the pre-constructed safety monitoring network includes: Constructing a convolutional neural network and using the convolutional neural network as the safety monitoring network.

3. The punching aluminum single panel processing safety warning method according to claim 1, characterized in that, The construction method of the loss function for controlling the learning of operation relationships includes: ； Among them, represents a sequence composed of job relation change descriptors formed by all corresponding key point pairs in two frames of punched aluminum single panel processing images, represents the sequence obtained by flattening the association graph, represents the sequence and the sequence correlation, and S represents the loss function for controlling job relation learning.

4. A punching aluminum single panel processing safety warning method according to claim 1, characterized in that, The obtaining method of the operation relationship change descriptor composed of the corresponding key point pairs in two frames of processed images of punched aluminum single plates includes: Performing segmentation processing on each frame of processed image of punched aluminum single plates to obtain the operator area and the machine area; Forming a key point pair by the key point in the operator area of a frame of processed image of punched aluminum single plates and the nearest key point in the machine area, obtaining the vector formed by the two key points in any key point pair, obtaining the vector formed by the corresponding key point pair in another frame of processed image of punched aluminum single plates, and denoting the difference vector of the vectors obtained from the two corresponding key point pairs in the two frames of processed images of punched aluminum single plates as the operation relationship change description vector based on this key point pair; multiplying the included angle between the operation relationship change description vector and the unit vector to the right horizontally by the modulus length to obtain the operation relationship change descriptor of this key point pair.

5. A punching aluminum single panel processing safety warning method according to claim 4, characterized in that, The performing of segmentation processing on each frame of processed image of punched aluminum single plates to obtain the operator area and the machine area includes: Inputting each frame of processed image of punched aluminum single plates into a pre-trained image segmentation network to obtain the operator area and the machine area in each frame of processed image of punched aluminum single plates.

6. The punching aluminum single board processing safety warning method according to claim 1, characterized in that The obtaining of the association graph formed by the feature maps of two frames of processed images of punched aluminum single plates includes: Performing superposition processing on the feature maps obtained from each frame of processed image of punched aluminum single plates to obtain the comprehensive feature map of each frame of processed image of punched aluminum single plates; Using the difference image between the comprehensive feature map of one frame of processed image of punched aluminum single plates and the comprehensive feature map of another frame of processed image of punched aluminum single plates as the association graph.

7. A safety warning method for the processing of punched aluminum single plates according to claim 1, characterized in that The achieving of safety warning for the processing of punched aluminum single plates includes: Inputting a newly collected processing video of punched aluminum single plates into the safety monitoring network after training to obtain a safety detection result. If the safety detection result indicates the existence of a safety hazard, then issue a warning.

8. A safety warning method for the processing of punched aluminum single plates according to claim 1, characterized in that, Flowing the association graph together with the feature map into the next link of the safety monitoring network.

9. A processing safety warning system for punched aluminum single plates, characterized in that, Including: A processor and a memory, the memory storing computer program instructions which, when executed by the processor, implement a punching aluminum single board processing safety warning method according to any one of claims 1-8.

Images