Stacked part identification method and device and electronic equipment

Abstract

The invention relates to the technical field of computers, in particular to a stacked part recognition method and device and electronic equipment, and the method comprises the steps that a part image collected at a target part feeding point position on a target production line is obtained, and the part image is collected for a part placed at the target part feeding point position; then, carrying out stack identification by a stack identification model according to the part image to obtain a stack identification result; if the stacked part identification result indicates that the overlapped parts exist at the target part feeding point position, sending stacked part prompt information to line body equipment responsible for the target part feeding point position on the target production line; through the method provided by the invention, whether part overlapping occurs at the target part feeding point or not can be automatically identified, and under the condition that part overlapping occurs at the target part feeding point, the part overlapping prompt information is automatically sent to the line body equipment to remind the line body equipment, so that correct operation of the target production line is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a stack piece identification method and device and electronic equipment. BACKGROUND

[0002] With the rapid development of intelligent manufacturing, automated production lines have been widely used. Through the automated production line, automatic machining and assembly of parts can be realized, greatly improving manufacturing efficiency.

[0003] In related technologies, a sensor is usually used to detect whether a part is correctly placed at a piece loading point, thereby effectively preventing part missing or part type assembly error, but it cannot detect the overlap of the same parts at the same piece loading point. Therefore, how to detect overlapping parts is a problem to be solved. SUMMARY

[0004] Therefore, the embodiments of the present application provide a stack piece identification method, device and electronic equipment, which can realize stack piece identification of parts at a target piece loading point.

[0005] The embodiments of the present application are implemented by adopting the following technical solutions: In a first aspect, the embodiments of the present application provide a stack piece identification method, comprising: acquiring a part image collected at a target piece loading point on a target production line, the part image being collected for a part placed at the target piece loading point; performing stack piece identification by a stack piece identification model according to the part image to obtain a stack piece identification result; and if the stack piece identification result indicates that there are overlapping parts at the target piece loading point, sending a stack piece prompt information to a line body device responsible for the target piece loading point on the target production line.

[0006] In a second aspect, the embodiments of the present application provide a stack piece identification device, comprising: an acquisition module configured to acquire a part image collected at a target piece loading point on a target production line, the part image being collected for a part placed at the target piece loading point; a stack piece identification module configured to perform stack piece identification by a stack piece identification model according to the part image to obtain a stack piece identification result; and an output module configured to send a stack piece prompt information to a line body device responsible for the target piece loading point on the target production line if the stack piece identification result indicates that there are overlapping parts at the target piece loading point.

[0007] In a third aspect, the embodiments of the present application provide a stack piece identification system, comprising: a line body device, a collection device and a control device; the collection device is configured to collect a part image of a part at a target piece loading point on a target production line; the control device is configured to perform stack piece identification according to the part image according to the above method; and the control device is configured to weld the part at the target piece loading point.

[0008] In a fourth aspect, an electronic device is provided, and the electronic device includes a processor, and a memory having computer instructions stored thereon, wherein the computer instructions, when executed by the processor, implement the method described above.

[0009] In a fifth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions, wherein the computer instructions, when executed by a processor, implement the method described above.

[0010] In a sixth aspect, a computer program product is provided, and the computer program product includes computer instructions, wherein the computer instructions, when executed by a processor, implement the method described above.

[0011] The method, device and electronic device provided by the embodiments of the present application can automatically identify whether the target feeding point has part overlapping, and automatically send the part overlapping prompt information to the line body device to remind the line body device when the target feeding point has part overlapping, thereby facilitating the correct operation of the target production line.

[0012] These aspects or other aspects of the present application will be made clearer in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some of the embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0014] Figure 1 A flowchart of a part overlapping identification method provided by an embodiment of the present application is shown.

[0015] Figure 2 A flowchart of the step S110 in the method provided by an embodiment of the present application is shown. Figure 1

[0016] Figure 3 ​A part image related to an embodiment of the present application is shown.

[0017] Figure 4 A schematic diagram of a training process of a stack identification model related to an embodiment of the present application is shown.

[0018] Figure 5 A schematic diagram of a part image set acquisition process related to an embodiment of the present application is shown. Figure 4

[0019] Figure 6 A schematic diagram of another stack identification method related to an embodiment of the present application is shown.

[0020] Figure 7 A schematic diagram of a stack identification system related to an embodiment of the present application is shown.

[0021] Figure 8 A schematic diagram of an application scenario of a stack identification system related to an embodiment of the present application is shown.

[0022] Figure 9a A schematic diagram of an execution process of a stack identification system related to an embodiment of the present application is shown.

[0023] Figure 9b A schematic diagram of a training process of a stack identification model related to an embodiment of the present application is shown.

[0024] Figure 10 A schematic diagram of a stack identification device related to an embodiment of the present application is shown.

[0025] Figure 11 A schematic diagram of an electronic device related to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in detail below with reference to the drawings, examples of which are shown in the accompanying drawings, in which the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are examples only, and are used to explain the present application, and should not be understood as limiting the present application.

[0027] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work fall within the scope of protection of the present application.

[0028] ​In the following description, the terms "first\second" and the like are merely intended to distinguish similar objects and do not represent a specific order or precedence of the objects. Understandably, the "first\second" can be interchangeable in specific order or precedence as long as it is allowed, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0029] "Multiple" mentioned herein refers to two or more. "And / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. In the following description, "some embodiments" or "some embodiments" are described as a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0030] For the convenience of understanding the present application, some terms will be explained first.

[0031] Putting: refers to the process of placing a part at a designated position on the production line; the designated position is also the part's putting point.

[0032] Stacking: refers to the state of two or more identical parts being stacked together.

[0033] In related technologies, the specific position of the part is usually detected by a sensor to effectively prevent missing parts or incorrect assembly of part types, but it cannot avoid repeated assembly of identical parts at the same putting point. Therefore, how to detect repeatedly assembled parts is a problem to be solved, and for this purpose, the present application provides a stacking recognition method, device and electronic equipment.

[0034] The present application will be described in detail below with reference to the embodiments.

[0035] Please refer to Figure 1 , Figure 1 A stacking recognition method related to an embodiment of the present application is given, which includes steps S110-S130: S110, acquiring a part image collected at a target putting point on a target production line, the part image being collected for a part placed at the target putting point.

[0036] In this application, the target production line can refer to a welding production line for welding parts, or a production line for other operations on parts (such as a stamping production line, a riveting production line, etc.).

[0037] There can be multiple feeding points on the target production line, each of which is used to place a part, and the parts placed in different feeding points can be the same or different; in this application, the target feeding point is the point that needs to be identified.

[0038] In some embodiments, the part image of the part at the target feeding point can be collected by the collection device, and then the part image is obtained from the collection device; the collection device is, for example, an industrial camera, a camera, etc.

[0039] In some embodiments, the collected part image can be multiple; the multiple part images are images collected from multiple perspectives of the part facing the target feeding point; the part image from multiple perspectives can more comprehensively describe the state of the part at the target feeding point, thereby facilitating subsequent stacking identification model to identify stacking according to the part image from multiple perspectives.

[0040] In some embodiments, please refer to Figure 2 , Figure 2 The flowchart of step S110 in the method provided by the application is shown, and step S110 includes steps S210-S220: Figure 1 S210, receiving the feeding completion signal of the target feeding point on the target production line.

[0041] In some embodiments, the feeding completion signal can be manually triggered by the production line personnel, for example, when the target feeding point is completed, the production line personnel manually presses the feeding completion button to trigger the feeding completion signal; of course, the feeding completion signal can be automatically triggered, for example, by detecting the part placement condition of the target feeding point through an infrared sensor, and when it is detected that the target feeding point has placed a part, it is considered that the feeding is completed, and the feeding completion signal is automatically triggered.

[0042] S220, in response to the feeding completion signal, image collection is performed facing the part at the target feeding point on the target production line, and a part image is obtained.

[0043] In some embodiments, in response to the feeding completion signal, the collection device can be called to perform image collection facing the part at the target feeding point on the target production line, and then the image collected by the collection device is uploaded to obtain the part image.

[0044] In the above embodiments, after receiving the feeding completion signal of the target feeding point, image collection is performed, which can ensure that the collected part image contains the part placed at the target feeding point, that is, to ensure that the collected part image is valid.

[0045] ​Meanwhile, the part image is collected by facing the part placed on the target part point, so the part image collected will inevitably include the image of the area where the part is located, so the part image can be used to identify the stacking state of the part.

[0046] Please refer to Figure 3 , Figure 3 Exemplarily, two part images are given. Figure 3 In a, the part image is a part image without overlapping parts, Figure 3 In b, the part image is a part image with overlapping parts; it can be seen that the overlapping parts and the non-overlapping parts are often highly similar in appearance, and the main difference is the thickness of the part, so the specific placement posture of the part needs to be determined to determine the collection angle of the part image; specifically, the collection angle is the angle that needs to cover the thickness of the part, so that the part image collected can reflect the characteristics of the part in thickness.

[0047] S120, identifying the stacking of the part by the stacking identification model according to the part image, to obtain a stacking identification result.

[0048] The stacking identification model is a model for identifying overlapping parts; it can be a convolutional neural network model.

[0049] The stacking identification result is used to indicate whether there are overlapping parts on the target part point.

[0050] In some embodiments, the complete part image can be input into the stacking identification model for stacking identification to obtain the stacking identification result.

[0051] In another embodiment, the part pixel area where the part is located in the part image can also be input into the stacking identification model for stacking identification to obtain the stacking identification result; in this case, before step S120 is performed, the stacking identification method can further include: performing part identification on the part image to obtain the part pixel area where the part is located in the part image.

[0052] At this time, step S120 can specifically include: identifying the stacking of the part by the stacking identification model according to the part pixel area, to obtain the stacking identification result.

[0053] In some embodiments, part identification can be achieved by edge detection on the part image; exemplarily, edge detection can be performed on the part image by OpenCV (Open Source Computer Vision Library), and the detected edge is most likely the contour of the part, and then the area framed by the edge is determined as the part pixel area where the part is located.

[0054] In some embodiments, the part image can be subjected to target detection by a target detection model, and the part pixel region in which the part is located in the part image can be determined according to the result of the target detection.

[0055] In the above embodiments, compared with the complete part image, the part pixel region only retains the image region in which the part is located, thereby avoiding the interference of the image region in which the part is not located in the part image on the stack identification.

[0056] In some embodiments, the stack identification model is a pre-trained neural network model, such as a convolutional neural network (CNN).

[0057] Please refer to Figure 4 , Figure 4 The training process of the stack identification model is given, including steps S310-S340: S310, obtaining a reference part image set and reference labels of each reference part image in the reference part image set; wherein the reference label of the reference part image is used to indicate the stack state of the sample part presented in the reference part image, and the stack state is a first state indicating that the sample part is an overlapping part, or a second state indicating that the sample part is not an overlapping part.

[0058] In some embodiments, the reference label of the reference part image is obtained by manual annotation by a training personnel according to the reference part image.

[0059] In some embodiments, the reference part image in the reference part image set is an image collected for a sample part on a sample part mounting point on a sample production line; further, the collection angle of the reference part image can be multiple, such as a front view angle, a side view angle, a pitch view angle, etc.; the reference part image set can include reference part images collected in multiple collection environments, such as a light illumination environment, a non-light illumination environment, etc.

[0060] Wherein, the sample part mounting point refers to the point where the sample part is placed; the sample production line refers to a production line including the sample part mounting point and used for working on the sample part.

[0061] In other embodiments, please refer to Figure 5 , Figure 5 The acquisition process of the reference part image set in the embodiment provided by the present application is shown in Figure 4 The acquisition of the reference part image can include steps S410-S430: S410, obtaining a first part image; the first part image is an image collected for a sample part on a sample part mounting point on a sample production line.

[0062] In some embodiments, the first part image can include images of the sample part captured from multiple perspectives facing the sample part on the sample part mounting point on the sample production line; the first part image can also include part images captured in multiple capture environments, such as a light illumination environment, a non-light illumination environment, and the like.

[0063] S420, performing image augmentation processing on the first part image to obtain a second part image; wherein the image augmentation processing includes at least one of image illumination adjustment and image rotation.

[0064] The image illumination adjustment can include brightness adjustment, contrast adjustment, saturation adjustment, and illumination angle change, etc.; the image rotation can include left-right flipping, up-down flipping, etc.

[0065] In other embodiments, the image augmentation processing can also include image cropping (such as cropping non-part regions in the first part image), image stretching, and the like processing methods, and one or more methods can be selected for image augmentation processing according to actual needs.

[0066] S430, adding the first part image and the second part image to the reference part image set.

[0067] In the above embodiments, the reference part image in the reference part image set includes the first part image captured facing the sample part on the sample part mounting point on the sample production line, and the second part image obtained by performing augmentation processing on the first part image, which enriches the diversity of the reference part image, so that the stacking recognition model trained using the reference part image has good robustness.

[0068] S320, performing stacking recognition on the reference part image by the stacking recognition model to obtain a stacking recognition result of the reference part image.

[0069] Specifically, the image features of the reference part image are extracted by the stacking recognition model, and then stacking probability prediction is performed according to the extracted image features; if the predicted stacking probability is greater than a probability threshold, a stacking recognition result indicating the existence of overlapping parts is output; if the predicted stacking probability is not greater than the probability threshold, a stacking recognition result indicating the non-existence of overlapping parts is output.

[0070] The stacking recognition result of the reference part image includes a first result indicating the existence of overlapping parts, and a second result indicating the non-existence of overlapping parts.

[0071] In some embodiments, the reference label of the reference part image can be further used to indicate that there are several overlapping parts in the case that the first state indicates that the sample part is an overlapping part; in this case, the stack identification result output by the stack identification model can further indicate that there are several overlapping parts on the basis of indicating that there are overlapping parts.

[0072] S330, calculate the model loss based on the stack identification result of each reference part image and the reference label of each reference part image.

[0073] In some embodiments, the model loss can be calculated by a loss function; for example, a KL divergence loss function, a cross-entropy loss function, etc.

[0074] S340, perform parameter adjustment on the stack identification model based on the model loss until a training end condition is reached.

[0075] The training end condition can be that the number of iterations of the stack identification model reaches an iteration threshold, or that the calculated model loss is less than a loss threshold.

[0076] S130, if the stack identification result indicates that there are overlapping parts at the target stacking point, send a stack prompt information to the line body device responsible for the target stacking point on the target production line.

[0077] The line body device refers to the device on the target production line; in this application, the line body device refers to the work device on the target production line for subsequent work (such as welding, stamping, etc.) on the parts at the target stacking point.

[0078] The stack prompt information is used to prompt the line body device that there are overlapping parts at the target stacking point; further, since there are overlapping parts at the target stacking point at this time, it belongs to stacking abnormality and cannot be welded, therefore, the stack prompt information can also prompt the line body device to stop welding the parts at the target stacking point.

[0079] In some embodiments, the stack prompt information can include the position information of the target stacking point and an error code for indicating the existence of overlapping parts; after the line body device receives the stack prompt information, it can alarm and display the position information of the target stacking point and the error code included in the stack prompt information according to the stack prompt information.

[0080] In some embodiments, after sending the stack prompt information to the line body device responsible for the target stacking point on the target production line, the stack prompt information, the part image, and the generation time of the stack prompt information can be associated and stored to generate an abnormal record, so as to facilitate subsequent fault tracing and troubleshooting according to the generated abnormal record.

[0081] The method provided in the application obtains a part image collected at a target feeding point on a target production line, the part image being collected for a part placed at the target feeding point. Apparently, since the part image is collected for the part, the part image must contain a region where the part is located, so that the state of the part at the target feeding point can be recognized according to the part image. Then, a stack identification model performs stack identification according to the part image to obtain a stack identification result. If the stack identification result indicates that there are overlapping parts at the target feeding point, a stack prompt information is sent to a line body device responsible for the target feeding point on the target production line. In the above process, automatic identification of whether there are overlapping parts at the target feeding point is realized, and in the case that there are overlapping parts at the target feeding point, a stack prompt information is sent to the line body device to remind the line body device, so as to facilitate correct operation of the target production line.

[0082] In some embodiments, referring to Figure 6 After step S120, the stack identification method can further include step S140: S140, if the stack identification result indicates that there are no overlapping parts at the target feeding point, a welding signal is sent to the line body device; the welding signal is used to prompt the line body device to weld the parts at the target feeding point.

[0083] In this case, the target production line is a production line involving a welding process, and the line body device can be a device for welding the parts at the target point.

[0084] It can be understood that the stack identification result indicating that there are no overlapping parts at the target feeding point means that the parts placed at the target feeding point are correct and can be welded, so that a welding signal can be sent to the line body device to prompt the line body device to weld the parts at the target feeding point.

[0085] In some embodiments, referring to Figure 7 , Figure 7 A system schematic diagram of a stack identification system provided by an embodiment of the application is shown, the stack identification system including a line body device 10, a collection device 20 and a control device 30; the line body device 10 is in communication connection with the control device 30, and the control device 30 is in communication connection with the collection device 20.

[0086] The collection device 20 is configured to collect a part image of a part at a target feeding point on a target production line; the control device 30 is configured to perform stack identification according to the part image according to the stack identification method provided by the application; and the line body device 10 is configured to process, assemble or the like the part at the target feeding point, for example, welding, painting, punching or the like.

[0087] In some embodiments, the line body device 10 can also send a loading completion signal to the control device 30 after detecting the completion of loading the part on the target loading point; the control device 30 can send a collection signal to the collection device 20 after receiving the loading completion signal, so that the collection device 20 collects images in response to the collection signal to obtain part images, and then the collection device 20 returns the collected part images to the control device 30, so that the control device 30 performs stack identification according to the part images according to the stack identification method provided in the present application.

[0088] In some embodiments, the line body device 10 can also send a loading completion signal to the control device 30 after detecting the completion of loading the part on the target loading point; the control device 30 can send a collection signal to the collection device 20 after receiving the loading completion signal, so that the collection device 20 collects images in response to the collection signal to obtain part images, and then the collection device 20 returns the collected part images to the control device 30, so that the control device 30 performs stack identification according to the part images according to the stack identification method provided in the present application. Figure 8 , Figure 8 The application scenario of the stack identification system is shown, which can include not only the line body device 10, the collection device 20 and the control device 30, but also the input device 40 and the power supply device 50.

[0089] The input device 40 can be used for a test personnel to edit a software program required for stack identification; the software program required for stack identification can include an image collection program, a stack identification model training program, and a stack identification program, etc. Further, the stack identification method of the present application can be realized by executing the edited software program.

[0090] In some embodiments, the image collection program can be used to pre-set the image collection area, the collection frequency, the collection quantity, the storage location of the part image, and other related contents related to image collection. For example, by running the image collection program, a preset quantity of part images can be automatically collected at a preset frequency, and then the collected part images are numbered in the order of collection time, and the part images and the corresponding numbers are stored in the designated memory, so that the test personnel can select the part images that can be used for stack identification model training from the collected part images.

[0091] In some embodiments, the stack identification model training program can be used to pre-set the related contents of model training; for example, automatically obtaining the part images required for training, automatically stopping training when the training end condition is reached, and evaluating the performance of the trained stack identification model, etc.

[0092] In some embodiments, the stack identification program can be executed to realize the stack identification method provided in the present application.

[0093] The power supply device 50 can supply power to the line body device 10, the collection device 20, the control device 30, and the input device 40; further, the power supply device 50 can include multiple specifications of output interfaces to meet the power demand of different devices.

[0094] In some embodiments, the line body device 10 can be a welding robot, the collection device 20 can be a CSI (Camera Serial Interface) camera, the control device 30 can be an embedded mainboard (such as a jetson mainboard), the input device 40 can be a computer, and the power supply device 50 can be a DC power adapter.

[0095] For example, taking a door production line in automobile body-in-white manufacturing as an example, the door part image is collected by an external camera, and the image is transmitted to a designated computer after collection. A development environment is configured on the computer, such as installing OpenCV (a cross-platform computer vision and machine learning software library) and pytorch (a deep learning framework) in a python programming environment. Based on OpenCV, part positioning, edge detection, and feature extraction are completed to obtain an image of a local feature, that is, a part pixel area of the present application. Then, at least 200 door part images are collected by the external camera, of which 100 are door part images without overlapping parts and 100 are door part images with overlapping parts. Of these, 160 are used to train the part recognition model (including 80 door part images without overlapping parts and 80 door part images with overlapping parts), and 40 (including 20 door part images without overlapping parts and 20 door part images with overlapping parts) are used to test the trained part recognition model. The door part image is used to train the part recognition model on the computer. By adjusting the model parameters and iterating multiple times, the recognition accuracy of the part recognition model is improved. The trained and improved part recognition model is imported into the JETSON board, and functions such as image collection, feature extraction, and result output are integrated. According to the Profinet protocol specification, communication between the production line device (i.e., the line body device of the present application) is realized, image collection-analysis determination-line communication is completed, and the complete control logic of the robot starting the welding program is realized.

[0096] In the above part recognition system, the part state of a certain part point in the automobile manufacturing industry can be automatically recognized and classified based on computer vision and part recognition model. Through this system, the accuracy and efficiency of part feeding are effectively improved, the need for manual intervention is reduced, the production process is optimized, and the error rate and operating cost are reduced.

[0097] In other embodiments, please refer to Figure 9a , Figure 9a An execution flow diagram of a part recognition system provided by an embodiment of the present application is shown, and the execution flow of the part recognition system is as follows: 101, collect images. Exemplarily, a part image can be collected by an external camera; further, a series of processing such as part positioning, edge detection, feature extraction, etc. can be realized by OpenCV to obtain a part pixel region in the part image.

[0098] 102, model training. Specifically, the image collected in step 101 is used to iteratively train the stack identification model until the training end condition is reached; wherein the stack identification model can be a CNN model.

[0099] 103, function integration. Specifically, the trained stack identification model is deployed on the JETSOIN board, and functions such as calling an external camera for image collection, calling OpenCV for image processing, and result output are integrated on the JETSOIN board, so that the stack identification method provided by the present application can be realized through the JETSOIN board.

[0100] 104, interfacing and communication. Specifically, the communication between each device in the stack identification system is established, for example, the communication between the external camera and the JETSOIN board, the communication between the JETSOIN board and the line body device, etc.

[0101] Among them, model training and function integration can be realized on a computer; specifically, a development environment is configured on the computer, that is, OpenCV and pytorch are installed in the python programming environment, then the part image collected by the external camera is used for image training, for example, 200 part images are collected, including 100 part images without overlapping parts and 100 images with overlapping parts, wherein 160 are used to train the stack identification model (including 80 part images without overlapping parts and 80 images with overlapping parts), and 40 (including 20 part images without overlapping parts and 20 images with overlapping parts) are used to test the trained stack identification model, a CNN model is built based on the image size, and after 120 iterations, the accuracy of the stack identification model reaches 95%.

[0102] In some embodiments, the stack identification model can be trained in a supervised training manner; specifically, before training the stack identification model with part images, reference labels need to be added to the part images; then, the stack identification model performs stack identification on the part images, outputs the predicted labels of the part images, calculates the model loss according to the reference labels and the predicted labels, and adjusts the stack identification model according to the model loss. Exemplarily, please refer to Figure 9b , Figure 9bA training schematic diagram of the stack identification model is shown, a part image with a reference label is input into the stack identification model, the stack identification model is used for stack identification on the part image, and a predicted label of the part image is obtained; wherein, the label 1 represents that there is no overlapping part, and the label 2 represents that there is overlapping part.

[0103] In some embodiments, the training module is further configured to obtain a first part image; the first part image is an image collected for a sample part on a sample feeding point on a sample production line; and the first part image is subjected to image augmentation processing to obtain a second part image; wherein, the image augmentation processing comprises at least one of image light adjustment and image rotation. Figure 10 , Figure 10 A schematic diagram of a stack identification device provided by an embodiment of the present application is shown, and the stack identification device 500 comprises: The acquisition module 510 is configured to acquire a part image collected at a target feeding point on a target production line, and the part image is collected for a part placed at the target feeding point.

[0104] The stack identification module 520 is configured to perform stack identification on the part image by using the stack identification model to obtain a stack identification result.

[0105] The output module 530 is configured to send stack prompt information to a line body device responsible for the target feeding point on the target production line if the stack identification result indicates that there is an overlapping part at the target feeding point.

[0106] In some embodiments, the output module 530 is further configured to send a welding signal to the line body device if the stack identification result indicates that there is no overlapping part at the target feeding point; and the welding signal is used to prompt the line body device to weld the part on the target feeding point.

[0107] In some embodiments, the stack identification device 500 further comprises a training module, the training module is configured to obtain a reference part image set and a reference label of each reference part image in the reference part image set; wherein, the reference label of the reference part image is used to indicate a stack state of a sample part presented in the reference part image, the stack state is a first state indicating that the sample part is an overlapping part, or a second state indicating that the sample part is not an overlapping part; the stack identification model is used to perform stack identification on the reference part image to obtain a stack identification result of the reference part image; a model loss is calculated based on the stack identification result of each reference part image and the reference label of each reference part image; and the stack identification model is subjected to parameter adjustment based on the model loss until a training end condition is reached.

[0108] In some embodiments, the training module is further configured to obtain a first part image; the first part image is an image collected for a sample part on a sample feeding point on a sample production line; the first part image is subjected to image augmentation processing to obtain a second part image; wherein, the image augmentation processing comprises at least one of image light adjustment and image rotation; and the first part image and the second part image are added to the reference part image set.

[0109] In some embodiments, the stack identification device 500 further comprises a part identification module, configured to perform part identification on the part image to obtain a part pixel region in which the part in the part image is located; and the stack identification module 520 is further configured to perform stack identification on the part pixel region according to the stack identification model to obtain a stack identification result.

[0110] In some embodiments, the acquisition module 510 is specifically configured to receive an unloading completion signal of a target unloading point on a target production line; and in response to the unloading completion signal, perform image acquisition on the part facing the target unloading point on the target production line to obtain the part image.

[0111] In some embodiments, the part image is a plurality of part images; and the plurality of part images are images collected from multiple perspectives of the part facing the target unloading point.

[0112] Figure 11 A structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown. The electronic device can be a terminal as described above, and is configured to implement the stack identification method provided by the present application. It should be noted that, Figure 11 The computer system 1300 of the electronic device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0113] As shown in Figure 11 The computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1302 or programs loaded from a storage portion 1308 into a random access memory (RAM) 1303, such as performing the methods in the above embodiments. In the RAM 1303, various programs and data required for system operation are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0114] The following components are connected to the I / O interface 1305: an input part 1306 including a keyboard, a mouse, a microphone, etc.; an output part 1307 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage part 1308 including a hard disk, etc.; and a communication part 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as necessary. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1310 as necessary, so that computer instructions read therefrom are installed in the storage part 1308 as necessary.

[0115] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising computer instructions. When the computer instructions are executed by a central processing unit (CPU) 1301, various functions defined in the system of the present application are performed.

[0116] The present application also provides a computer readable storage medium, which stores computer instructions, and the computer instructions are executed by a processor to implement the method in any of the method embodiments above.

[0117] It should be noted that the computer readable storage medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the above two. The computer readable storage medium may, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or instrument, or any combination of the above. More specific examples of computer readable storage medium can include, but are not limited to: electrical connection with one or more conductive wires, portable computer disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), flash memory, optical fiber, portable compact disk read only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing programs that can be used or combined with instruction execution system, device or instrument. In the present application, the computer readable signal medium can include data signals carrying computer readable program code in the baseband or as part of a carrier wave. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals or any suitable combination of the above. The computer readable signal medium can also be any computer readable storage medium other than the computer readable storage medium, which can send, propagate or transmit programs for use by or in combination with instruction execution system, device or instrument. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination of the above.

[0118] In the embodiments of the present application, the term "module" or "unit" refers to computer instructions or part of computer instructions with predetermined functions, and works with other related parts to achieve predetermined targets, and can be implemented entirely or partially by using software, hardware (such as processing circuit or memory) or combination thereof, similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit of the function of the module or unit.

[0119] The above merely describes the preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed with the preferred embodiments as above, it is not intended to limit the present application, and any person skilled in the art can make some more changes or modifications to the equivalent embodiments with the disclosed technical content, as long as the changes or modifications do not deviate from the technical solution of the present application. Any brief modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application still falls within the scope of the technical solution of the present application.

Claims

1. A stack identification method characterized by, The method comprises: acquiring a part image collected at a target feeding point on a target production line, the part image being collected from a part placed at the target feeding point; performing, by a stack identification model, stack identification on the part image to obtain a stack identification result; if the stack identification result indicates that there are overlapping parts at the target feeding point, sending a stack prompt information to a line equipment responsible for the target feeding point on the target production line.

2. The method of claim 1, wherein, After the stack identification model performs stack identification on the part image to obtain the stack identification result, the method further comprises: if the stack identification result indicates that there are no overlapping parts at the target feeding point, sending a welding signal to the line equipment; the welding signal is used to prompt the line equipment to weld the part at the target feeding point.

3. The method of claim 1, wherein, The training process of the stack identification model comprises: acquiring a reference part image set and a reference label of each reference part image in the reference part image set; wherein the reference label of the reference part image is used to indicate the stack state of a sample part presented in the reference part image, and the stack state is a first state indicating that the sample part is an overlapping part, or a second state indicating that the sample part is not an overlapping part; performing, by the stack identification model, stack identification on the reference part image to obtain a stack identification result of the reference part image; calculating a model loss based on the stack identification result of each reference part image and the reference label of each reference part image; adjusting parameters of the stack identification model based on the model loss until a training end condition is reached.

4. The method of claim 3, wherein, The acquisition of the reference part image set comprises: acquiring a first part image; the first part image is an image collected from a sample part at a sample feeding point on a sample production line; performing image augmentation processing on the first part image to obtain a second part image; wherein the image augmentation processing comprises at least one of image light adjustment and image rotation; adding the first part image and the second part image to the reference part image set.

5. The method according to any of claims 1 to 4, characterized in that Before the stack identification model performs stack identification on the part image to obtain the stack identification result, the method further comprises: performing part identification on the part image to obtain a part pixel region where the part in the part image is located; the stack identification model performs stack identification on the part pixel region to obtain the stack identification result. The acquisition of the part image collected at the target feeding point on the target production line comprises:

6. The method according to any one of claims 1 to 4, characterized in that, receiving a part feeding completion signal of the target feeding point on the target production line; in response to the part feeding completion signal, performing image collection on the part at the target feeding point on the target production line to obtain the part image. The part image is multiple; the multiple part images are images collected from the part at the target feeding point from multiple perspectives.

7. The method according to any one of claims 1 to 4, characterized in that, The method comprises:

8. A stack identification device, characterized by ​ An acquisition module is configured to acquire a part image collected by a target feeding point on a target production line, the part image being collected by a camera facing a part placed on the target feeding point; A stack identification module is configured to perform stack identification according to the part image by using a stack identification model to obtain a stack identification result; An output module is configured to send stack prompt information to a line equipment responsible for the target feeding point on the target production line if the stack identification result indicates that there are overlapping parts on the target feeding point.

9. An electronic device, comprising: comprise: a processor; a memory, the memory storing computer instructions, the computer instructions being executed by the processor to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, the computer instructions being executed by the processor to implement the method of any one of claims 1-7.

Images