A packing line disordered line alarm method, device, equipment and storage medium

Abstract

The application provides a packing line disorder alarm method, device, equipment and storage medium, the method obtains the target environment image of the target packing station; the target environment image is input into the packing feature labeling model to obtain the packing line mark and the packing station mark bit mark, the initial environment image of the target packing station is required to train the packing feature labeling model; the color channels of the initial environment image are converted to each other to generate a converted environment image; the initial environment image and the converted environment image are determined as sample images and are labeled to generate a sample data set; and the trained model is determined as the packing feature labeling model; the packing line state of the target packing station is determined based on the number of packing line marks of the packing line mark and the number of packing station mark bit marks of the packing station mark bit mark, and alarm information is sent out; the image acquisition and image processing involved in the application are completed by machine equipment, the accuracy of judgment is improved, and the labor cost is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a packing line disorder alarm method, device, equipment and storage medium. BACKGROUND

[0002] In the production process of steel products, the packing process is an important step of wire rod production, and there are multiple packing machines in the packing station, and each packing machine needs corresponding wire rods for packing. In the process of wire rod feeding, there is a certain probability that the wire rods will be entangled, and once the wire rods are entangled, the feeding will be blocked, which will cause the wire rods to be pulled apart. Therefore, in actual production, it is particularly important to timely detect the entanglement of the wire rods and handle it. The current market generally uses experienced workers to manually identify the packing line disorder, but since multiple production lines need to run for a long time, a large amount of manpower needs to be invested to complete the packing line disorder identification, and the manual identification is limited by the subjective consciousness and production experience of the workers, and the judgment conclusion has uncertainty and the accuracy cannot be estimated.

[0003] Therefore, in the whole packing process, intelligent monitoring means needs to be used to detect whether the packing line is entangled in real time, and an alarm information needs to be sent in time when the wire rods are entangled, so as to save more manpower cost and improve the accuracy of the judgment conclusion. SUMMARY

[0004] In view of the shortcomings of the prior art, the present application provides a packing line disorder alarm method, device, equipment and storage medium to solve the above technical problems.

[0005] The present application provides a packing line disorder alarm method, which comprises: acquiring a target environment image of a target packing station; inputting the target environment image into a packing feature labeling model to obtain a packing line label and a packing station mark label, the training mode of the packing feature labeling model comprising: acquiring an initial environment image of the target packing station, the initial environment image comprising red, green and blue three primary color channels; converting the red, green and blue three primary color channels into each other, including interchanging the brightness value of the red color channel with the brightness value of the blue color channel to generate a converted environment image; determining the initial environment image and the converted environment image as sample images, labeling the packing line features and the packing station mark features in the sample images to generate a sample data set; training an initial labeling model based on the sample data set, and determining the trained initial labeling model as the packing feature labeling model; determining the packing line state of the target packing station based on the number of packing line labels of the packing line label and the number of packing station mark labels of the packing station mark label; and issuing an alarm information when the packing line state of the target packing station is a disorder state.

[0006] In an embodiment of the present application, the obtaining of the initial environment image of the target packing station further comprises: determining a region of interest of the initial environment image based on the initial environment image of the target packing station; determining a region outside the region of interest in the initial environment image as a non-region of interest, and filtering the non-region of interest.

[0007] In an embodiment of the present application, the labeling of the packing line feature and the packing station marker feature in the sample image comprises: performing rotational labeling of the packing line feature and the packing station marker feature in the sample image by a rectangular box, the rectangular box comprising a non-standard rectangular box; correcting the non-standard rectangular box to a standard rectangular box by a minimum circumscribed rectangle; eliminating abnormal angle loss values at boundaries based on a ring smoothing label method to obtain an angle prediction result of the packing feature labeling model; and detecting the packing line and the packing station marker with a rotation attribute within a preset angle range based on the angle prediction result to obtain a packing line feature labeling box and a packing station marker feature labeling box.

[0008] In an embodiment of the present application, after determining the packing line state of the target packing station based on the number of packing line identifications and the number of packing station marker identifications, the method further comprises: if the packing line state of the target packing station is a disorderly line state, determining that the feeding state of the packing machine is abnormal, the abnormal feeding state of the feeding machine comprising a packing line missing state and a packing line winding state; the packing machine is arranged at the target packing station and is used for packing products by using a packing line, and the packing station marker is arranged at the packing machine or the target packing station.

[0009] In an embodiment of the present application, determining that the feeding state of the packing machine is abnormal comprises: determining the positions of each packing line based on the packing line identification, and if the positions of at least two packing lines cross, determining that the abnormal feeding state is a packing line winding state; and if the positions of any two packing lines do not cross, determining that the abnormal feeding state is a packing line missing state.

[0010] In an embodiment of the present application, inputting the target environment image into the packing feature labeling model to obtain packing station identification and packing station marker identification comprises: inputting the target environment image into the packing feature labeling model to identify the packing line feature and the packing station marker feature in the target environment image to obtain a packing line labeling box and a packing station marker labeling box; and determining the packing line labeling box as a packing line identification and determining the packing station marker labeling box as a packing station marker identification.

[0011] In an embodiment of the present application, determining the packing line state of the target packing station based on the number of packing line identifiers and the number of packing station flag identifiers identified from the packing line identifiers comprises: determining the number of packing line identifiers and the number of packing station flag identifiers based on the packing line identifiers and the packing station flag identifiers; if the number of packing line identifiers is equal to the number of packing station flag identifiers, determining that the packing line state of the target packing station is a normal state; and if the number of packing line identifiers is less than the number of packing station flag identifiers, determining that the packing line state of the target packing station is a disorderly line state.

[0012] In an embodiment of the present application, issuing an alarm information based on the disorderly line state comprises: obtaining a plurality of frames of environment images of the target station, and detecting the packing line state of the plurality of frames of the target station; when it is detected that the packing line state of the plurality of frames of the target station is a disorderly line state, determining the number of frames of disorderly line state of the plurality of frames of the target station; and if the number of frames of disorderly line state is greater than or equal to a preset safety frame number threshold, issuing an alarm information.

[0013] The present application provides a packing line disorderly line alarm device, which comprises: an image acquisition module configured to obtain a target environment image of a target packing station; an identifier determination module configured to input the target environment image into a packing feature labeling model to obtain a packing line identifier and a packing station flag identifier; a state determination module configured to determine a packing line state of the target packing station based on the number of packing line identifiers and the number of packing station flag identifiers identified from the packing line identifiers; and an alarm module configured to issue an alarm information when the packing line state of the target packing station is a disorderly line state.

[0014] The present application provides an electronic device, which comprises: one or more processors; and a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the packing line disorderly line alarm method as described above.

[0015] The present application provides a computer readable storage medium, which is characterized in that a computer program is stored thereon, which, when executed by a processor of a computer, causes the computer to execute the packing line disorderly line alarm method as described above.

[0016] Beneficial effects: the packaging line alarm method, device, equipment and storage medium provided by the application, by acquiring the target environment image of the target packaging station, training the packaging feature labeling model based on the target environment image, and performing mask processing on the target environment image during model training to improve the inference speed of the model, and performing color channel change processing on the obtained environment image to obtain a new environment image, and taking the environment image obtained by color channel transformation and the environment image obtained by initial acquisition as sample data to improve the generalization ability of the model, and identifying the packaging line label and the packaging station flag label in the target environment image through the packaging feature model, and judging the state of the packaging line by comparing the number of packaging line labels and the number of packaging station flag labels, so as to issue an alarm according to the disordered state of the packaging line; the image acquisition, image processing and final conclusion involved are completed by machine equipment, so that the accuracy of judgment is greatly improved, and the labor cost is effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0017] The drawings incorporated into the specification and forming a part thereof, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application. It is apparent that the drawings described below are only some embodiments of the present application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art. In the drawings:

[0018] Figure 1 is a system architecture schematic diagram of the wire field packaging disordered line alarm system according to an exemplary embodiment of the present application;

[0019] Figure 2 is a packaging disordered line alarm identification step flow chart according to an exemplary embodiment of the present application;

[0020] Figure 3 is a packaging station schematic diagram according to an exemplary embodiment of the present application;

[0021] Figure 4 is a block diagram of the packaging disordered line alarm device according to an exemplary embodiment of the present application;

[0022] Figure 5 shows the structure of a computer system of an electronic device suitable for implementing embodiments of the present application. DETAILED DESCRIPTION

[0023] The present application can be implemented or applied in other different specific embodiments, and various modifications or changes can be made to the details based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, but not for limiting the protection scope of the present application.

[0024] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, but not the number, shape and size of the components when actually implemented. The actual implementation of each component can be a random change, and the component layout pattern can be more complex.

[0025] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, the known structures and devices are shown in the form of block diagrams rather than in the form of details, to avoid making the embodiments of the present application difficult to understand.

[0026] Figure 1 is a schematic diagram of a wire field packaging and disordering wire alarm system shown by an exemplary embodiment of the present application.

[0027] Referring to Figure 1 As shown in the figure, the system architecture can include an image acquisition device 101 and a computer device 102. The computer device 102 can be at least one of a desktop graphic processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. The environment image of the packaging station is acquired by the image acquisition device, and then the related technical personnel can use the computer device 102 to realize the processing of the environment image, so as to obtain the conclusion whether the packaging line of the packaging station exists disordering wire and whether it needs to alarm.

[0028] Schematically, the image acquisition device 101 first acquires the environment image of the target packaging station, and transmits the environment image to the computer device 102. The computer device 102 receives the environment image of the target packaging station, trains a packaging feature labeling model based on the environment image, then inputs the target environment image into the packaging feature labeling model to obtain packaging line identification and packaging station flag identification, and then judges the disordering wire condition of the packaging line of the target packaging station by comparing the number of packaging line identification and the number of packaging station flag identification, and sends an alarm information based on the disordering wire condition.

[0029] Figure 2 is a packing line wire alarm step flow chart shown in an exemplary embodiment of the application.

[0030] As Figure 2 shown, in an exemplary embodiment, the packing line wire alarm method comprises at least steps S210 to S240, which are described in detail as follows:

[0031] Step S210, obtaining a target environment image of a target packing station.

[0032] It should be understood that a fixed camera is provided at each packing station to collect the working status of each packing station. The fixed camera provided at the packing station collects the environment image of the target packing station. One or more cameras are provided around the target station, and the angle of the collected environment image can be adjusted by adjusting the angle of the camera to obtain multiple environment images including target packing station information, of which the image including 360 degrees of the target packing station is the best.

[0033] In an embodiment of the application, multiple 2073600-pixel high-definition camera devices are provided at different angles of the target packing station, and the camera devices are matched with the packing station, so that the relevant camera devices can collect pictures of the working conditions of each station in the scene. Based on the camera devices provided at the packing station, a 1920x1080 high-resolution environment image is collected to ensure that the identification labeling model can extract accurate image information for training to improve the accuracy of the identification labeling model.

[0034] Step S220, inputting the target environment image into a packing feature labeling model to obtain a packing line identification and a packing station mark identification. The training method of the packing feature labeling model comprises: obtaining an initial environment image of a target packing station, the initial environment image including red, green and blue color channels; converting the red, green and blue color channels to each other, including interchanging the brightness value of the red color channel with the brightness value of the blue color channel to generate a converted environment image; determining the initial environment image and the converted environment image as sample images, labeling the packing line features and the packing station mark features in the sample images to generate a sample data set; training the initial labeling model based on the sample data set, and determining the trained initial labeling model as the packing feature labeling model.

[0035] It should be understood that in the present application, a packaging feature labeling model is first trained according to the environment image of the target packaging station, and the model training steps are as follows: obtaining an initial environment image of the target packaging station, the initial environment image including red, green and blue color channels; converting the red, green and blue color channels to each other, including interchanging the brightness value of the red color channel with the brightness value of the blue color channel to generate a converted environment image; determining the initial environment image and the converted environment image as sample images, labeling the packaging line features and packaging station marker features in the sample images to generate a sample data set; training the initial labeling model based on the sample data set, and determining the trained initial labeling model as the packaging feature labeling model.

[0036] In the training process of the feature labeling model, obtaining the initial environment image of the target packaging station further includes: determining a region of interest of the initial environment image based on the initial environment image of the target packaging station; determining a region outside the region of interest in the initial environment image as a non-region of interest, and filtering the non-region of interest.

[0037] Since the environment image includes not only the packaging station marker and the packaging line, but also complex lighting conditions and human image, but the marker of the packaging station is a triangular structure with distinctive features on the packaging machine, and the packaging line is a long strip-shaped line segment, both of which are very easy to identify and classify, so the region where the packaging line and the packaging station marker are located in the environment image can be artificially defined as the region of interest, and the remaining part is the non-region of interest, and the non-region of interest is filtered out, thereby avoiding false detection and improving the model inference speed.

[0038] In an embodiment of the present application, a target environment image S is obtained, the image is divided into two parts S1 and S2, and the S1 region is the target region including the packaging line and the packaging station marker, and the S2 region includes other image information such as worker image information, the image S1 part is defined as the region of interest, and the S2 part is defined as the non-region of interest, and the S2 region is filtered.

[0039] Training the packaging feature labeling model first needs to obtain an environment image as a sample data set. Since the original environment image collected is limited, the original environment image collected can be processed by channel change or rotation to obtain a new environment image, and the new environment image obtained and the original environment image collected are used as sample images for model training together to increase the sample data set for model training, thereby improving the generalization ability of the model.

[0040] In an embodiment of the present application, by fixing a camera device in the working environment of the packing station, a plurality of environment images of the target station are collected, the collected environment image information is taken as a first image data set, and the collected environment images are changed in channel, that is, the brightness values of the respective three primary color channels of the original environment images are first extracted to obtain a red color channel brightness value X, a green color channel brightness value Y, and a blue color channel brightness value Z, the red color channel brightness value is converted with the blue color channel brightness value to obtain a new environment image with a red color channel brightness value Z, a green color channel brightness value Y, and a blue color channel brightness value X, the new environment image is taken as a second image data set, and the first image data set and the second image data set are merged for processing and collectively used as a sample data set required for model training.

[0041] In an embodiment of the present application, the obtained environment images can also be processed by rotation, that is, each environment image includes a packing line image and a packing station marker image, any environment image M is subjected to matting to obtain a corresponding packing line image and a packing station marker image N, the N in the image M is rotated by R degrees in the clockwise direction to obtain a new environment image M1, and all M1s are merged together as a third image data set, which is part of the sample image data set.

[0042] After obtaining the sample data for model training, the sample images are labeled.

[0043] The labeling of the packing line features and the packing station marker features in the sample images includes: rotating labeling of the packing line features and the packing station marker features in the sample images by a rectangular frame, the rectangular frame including a non-standard rectangular frame; correcting the non-standard rectangular frame to a standard rectangular frame by a minimum circumscribed rectangle, and eliminating abnormal angle loss values at boundaries based on a ring smoothing label method to obtain an angle prediction result of a packing feature labeling model; and detecting the packing line and the packing station marker with a rotation attribute within a preset angle range based on the angle prediction result to obtain a packing line feature labeling frame and a packing station marker feature labeling frame.

[0044] It should be understood that due to the visual effect of near large and far small and the rotation attribute, a plurality of packing lines in the field of view of the camera will have a large amount of overlap, and a large amount of missed detection may occur with a traditional non-rotation detection algorithm. To avoid such missed detection, an angle prediction result is added at the output end of the model, and a high-precision rotation target detection model is innovatively developed.

[0045] In one embodiment of the present application, the angle periodicity problem is solved by adopting a circular smooth label (CSL) to increase the error tolerance between adjacent angles, and the formula of the circular smooth label method is as follows:

[0046]

[0047] wherein g(x) is a window function, r is the radius of the window function, and theta is the angle of the current bounding box.

[0048] It should be understood that the initial environment image collected has diverse environmental information, including complex lighting conditions and human information, so a data set based on rotation labeling is needed to be made for the training of the deep learning model, so as to obtain the packaging line identification information and the packaging station mark identification information of the target packaging station. The standard rectangular frame involved is a parallelogram with one right angle, and the non-standard rectangle is other quadrilaterals that are not rectangles.

[0049] In an embodiment of the present application, the packaged station images obtained by shooting in a specific industrial scene are annotated, learned, detected, and the target boxes of both (i.e., the packaging line annotation box and the packaging station marker annotation box) are obtained. The position information of the packaging line and the packaging station marker in the image is detected by the target box, and the information is recorded and made into a packaging station rotation dataset, which is divided into three parts: training set, test set, and validation set. The data of the training set is used to train the packaging line and packaging station marker target detection model (i.e., the packaging feature annotation model). When the detection model is trained, the effective information that can be used for training after image annotation includes image basic attributes and annotation information. Image basic attributes include: filename-file name, width-width, height-height, and depth-image depth. Annotation information includes: Xmin, Ymin, Xmax, Ymax, which represent the left upper corner horizontal coordinate, the left upper corner vertical coordinate, the right lower corner horizontal coordinate, and the right lower corner vertical coordinate of each target box in the image, respectively; class, which is the category of the target object. The packaging line and packaging station marker in the target box range of each packaging station target environment image in the training set image are extracted by a deep learning network, and finally the rotation detection model of the packaging line and the marker (i.e., the packaging feature annotation model) is obtained. In this embodiment, the YOLOV5 neural network is selected, and other models such as transformer, RNN (LSTM), Faster-RCNN, R3det, etc. can also be selected. When the classification model is trained, the effective information that can be used for training after image annotation includes image basic attributes and annotation information. Image basic attributes include: filename-file name, width-width, height-height, and depth-image depth. Annotation information includes: Xmin, Ymin, Xmax, Ymax, which represent the left upper corner horizontal coordinate, the left upper corner vertical coordinate, the right lower corner horizontal coordinate, and the right lower corner vertical coordinate of each target box in the image, respectively; class, which is the category of the target object, which is divided into two categories: packaging line or marker, both of which have a rotation attribute. By learning the target features and the category of the target box range in each packaging station training set image, a packaging station target classification model is finally obtained.

[0050] It should be understood that the packing line and the packing station marker in the target environment image are labeled with the packing line identifier and the packing station marker identifier respectively by the above packaging feature labeling model, the number of the packing line identifier is the number of the packing line, and the number of the packing station marker identifier is the number of the packing station marker, so that the number of the packing line identifier and the number of the packing station marker identifier are obtained, that is, the number of the packing line and the number of the packing station marker are obtained, and the packing line disorder state of the packing station can be determined by comparing the number of the packing line identifier and the number of the packing station marker identifier, so that the alarm can be implemented according to the packing line disorder state.

[0051] In step S230, the packing line state of the target packing station is determined based on the number of the packing line identifier and the number of the packing station marker identifier.

[0052] The target environment image is input into the packaging feature labeling model to obtain the packing station identifier and the packing station marker identifier, including: inputting the target environment image into the packaging feature labeling model, identifying the packing station marker feature and the packing line feature in the target environment image to obtain the packing line labeling box and the packing station marker labeling box; determining the packing line labeling box as the packing line identifier and determining the packing station marker labeling box as the packing station marker identifier.

[0053] The packing line state of the target packing station is determined based on the number of the packing line identifier and the number of the packing station marker identifier, including: determining the number of the packing line identifier and the number of the packing station marker identifier based on the packing line identifier and the packing station marker identifier; if the number of the packing line identifier is equal to the number of the packing station marker identifier, it is determined that the packing line state of the target packing station is a normal state; if the number of the packing line identifier is less than the number of the packing station marker identifier, it is determined that the packing line state of the target packing station is a disorder state.

[0054] Figure 3 It is a packaging station schematic diagram shown in an exemplary embodiment of the present application.

[0055] As Figure 3As shown, the whole image is part of a target packing station, which includes packing station sign, packing line and packing machine. A, B and C with triangular structure are packing station signs, a, b and c with linear structure are packing lines, and the area connecting the packing station and the packing line is part of the packing machine. In the packing station as shown, there are multiple packing station signs and multiple packing lines, and each packing station sign corresponds to a packing line. The positions of A, B and C are the packing station signs, and the positions of a, b and c are the packing lines. Packing lines a and b are not broken, and packing line c is broken. In the process of image recognition and labeling, a and b are normally recognized and labeled, while c cannot be successfully labeled. Therefore, in the target environment image, when the packing line state of the packing station is normal, the number of packing station sign identifiers is equal to the number of packing line identifiers; when the packing line state of the packing station is a mess, the number of packing station sign identifiers is greater than the number of packing line identifiers. Since the packing line is wound, it will cause the packing line to break, so when the packing line is broken, the packing line is in a mess state.

[0056] In an embodiment of the present application, the obtained target environment image is input into a packing feature labeling model, so as to obtain the number of packing line feature identifiers K1 and the number of packing station sign feature identifiers K2 in the target environment image. By comparison, K1 < K2 is obtained, and it is determined that the target packing station is in a mess state.

[0057] In another embodiment of the present application, the obtained target environment image is input into a packing feature labeling model, so as to obtain the number of packing line feature identifiers K3 and the number of packing station sign feature identifiers K4 in the target environment image. By comparison, K3 = K4 is obtained, and it is determined that the target packing station is in a normal state.

[0058] Step S240, when the packing line state of the target packing station is in a mess state, an alarm information is sent out.

[0059] The alarm information based on the mess state includes: obtaining multiple frames of environment images of the target station, and detecting the packing line state of the multiple frames of target station; when it is detected that the packing line state of the multiple frames of target station is in a mess state, determining the number of mess frames of the packing line state of the multiple frames of target station; if the number of mess frames is greater than or equal to a preset safety frame number threshold, an alarm information is sent out.

[0060] In one embodiment of the present application, when the packing line of a certain packing station is in a disorder state, the preset safety frame number threshold for issuing an alarm is N1. When it is determined that the packing line state of the target packing station is in a disorder state, N frames of images are collected based on the current time, and the N frames of images are labeled to confirm that the packing station environment information in the N frames of images indicates that the packing line disorder state of the packing station is in a disorder state, and N is greater than N1. Then, the controller of the packing station issues an alarm information.

[0061] After determining the packing line state of the target packing station based on the packing line identification quantity and the packing station flag identification quantity, the following steps are further included: if the packing line state of the target packing station is in a disorder state, it is determined that the feeding state of the packing machine is abnormal, and the feeding abnormal state of the feeding machine includes a packing line missing state and a packing line winding state; the packing machine is arranged at the target packing station and is used for packing products with the packing line, and the packing station flag is arranged on the packing machine or the target packing station.

[0062] Determining that the feeding state of the packing machine is abnormal includes: determining the positions of each packing line based on the packing line identification, and if the positions of at least two packing lines cross, it is determined that the feeding abnormal state is a packing line winding state; and if the positions of any packing line do not cross, it is determined that the feeding abnormal state is a packing line missing state.

[0063] It should be understood that the target packing station is a spatial range, which includes the packing machine, the packing line and the packing station flag. The packing station flag can be arranged on the packing machine or the packing station, which is not limited herein.

[0064] In one embodiment of the present application, the packing station identification quantity and the packing line identification quantity of the target environment image are obtained based on the above packing feature labeling model, so as to determine that the packing line state of the target packing station is in a disorder state, and the feeding state of the packing machine of the packing station is determined to be in an abnormal state based on the disorder state of the packing line. The position information of each packing line is determined based on the packing line identification, and the position information of the plurality of packing lines is analyzed to obtain that the positions of the packing lines cross each other, so as to determine that the packing line of the packing station is wound, i.e. the feeding abnormal state of the packing machine is a packing line winding state.

[0065] In an embodiment of the present application, the number of packaging station identifiers and the number of packaging line identifiers of the target environment image are obtained based on the above packaging feature labeling model, so as to determine that the packaging line state of the target packaging station is a disorderly line state, and determine that the feeding state of the packaging machine of the packaging station is an abnormal feeding state based on the disorderly line state of the packaging line. The position information of each packaging line is determined based on the packaging line identifier, and the position information of the plurality of packaging lines is analyzed to obtain that each packaging line is independent of each other without intersection, so as to determine that the packaging line of the packaging station is not wound, that is, the abnormal feeding state of the packaging machine is a packaging line missing state.

[0066] Figure 4 is a block diagram of the packaging line disorderly line alarm device shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 the implementation environment shown in the figure and specifically configured in the intelligent terminal 102. The device can also be applied to other exemplary implementation environments and specifically configured in other devices, and the present embodiment does not limit the implementation environment to which the device is applied.

[0067] As Figure 4 shown, the exemplary packaging line disorderly line alarm device includes an image acquisition module 410, an identifier determination module 420, a state determination module 430, and an alarm module 440.

[0068] The image acquisition module 410 is configured to acquire a target environment image of a target packaging station; the identifier determination module 420 is configured to input the target environment image into a packaging feature labeling model to obtain a packaging line identifier and a packaging station identifier; the state determination module 430 is configured to determine a packaging line state of the target packaging station based on the number of packaging line identifiers of the packaging line identifier and the number of packaging station identifiers of the packaging station identifier; and the alarm module 440 is configured to issue an alarm information when the packaging line state of the target packaging station is a disorderly line state.

[0069] It should be noted that the packaging line disorderly line alarm device provided in the above embodiment and the packaging line disorderly line alarm method provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiment, which will not be described here. The packaging line disorderly line alarm device provided in the above embodiment can be completed by different functional modules according to the above functions in actual application, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions, and this is not limited herein.

[0070] Embodiments of the present application also provide an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the method for alarm of a packed line provided in each of the above embodiments.

[0071] Figure 5 A structural diagram of a computer system of an electronic device suitable for implementing embodiments of the present application is shown. It should be noted that, Figure 5 The computer system 500 of the electronic device shown is only an example and should not impose any limitation on the functions and applicable scope of embodiments of the present application.

[0072] As Figure 5 shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or programs loaded from a storage portion 508 into a random access memory (RAM) 503, such as performing the methods in the above embodiments. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0073] The following components are connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk, and the like; and a communication portion 509 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 510 as necessary, so that a computer program read therefrom is installed into the storage portion 508 as necessary.

[0074] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising computer programs for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 801, various functions defined in the system of the present application are executed.

[0075] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which the computer readable computer program is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transfer the program for use by or in connection with the instruction execution system, apparatus or device. The computer program contained on the computer readable medium can be transmitted in any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination of the above.

[0076] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0077] The units described in the embodiments of the present application can be implemented by software, or by hardware, or by a combination of software and hardware. The names of the units described are merely intended to represent the functions of the units, and are not intended to limit the units themselves.

[0078] Another aspect of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor of a computer, and causes the computer to perform the packing line wire dislocation alarm method as described above. The computer readable storage medium can be included in the electronic device as described in the embodiments above, or can exist separately and not be assembled into the electronic device.

[0079] Another aspect of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the packing line wire dislocation alarm method provided in the embodiments above.

[0080] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and scope of the present application should be covered by the claims of the present application.

Claims

1. A method for alarming tangled wires in packaging cables, characterized in that, The method includes: Obtain the target environment image of the target packaging station; The target environment image is input into the packing feature annotation model to obtain packing line identifiers and packing station marker identifiers. The training method of the packing feature annotation model includes: acquiring an initial environment image of the target packing station, the initial environment image including red, green and blue primary color channels; converting the red, green and blue primary color channels to each other, including swapping the brightness values ​​of the red color channel and the blue color channel to generate a converted environment image; determining the initial environment image and the converted environment image as sample images, annotating the packing line features and packing station marker features in the sample images to generate a sample dataset; training the initial annotation model based on the sample dataset, and determining the trained initial annotation model as the packing feature annotation model; The annotation of the packing line features and packing station marker features in the sample image includes: rotating and annotating the packing line features and packing station marker features in the sample image using a bounding box, the bounding box including non-standard bounding boxes; correcting the non-standard bounding boxes to standard bounding boxes using a minimum bounding rectangle; eliminating abnormal angle loss values ​​at the boundary based on a circular smoothing labeling method to obtain the angle prediction result of the packing feature annotation model; and detecting packing lines and packing station markers with rotation attributes within a preset angle range based on the angle prediction result to obtain packing line feature annotation boxes and packing station marker feature annotation boxes. The packing line status of the target packing station is determined based on the number of packing line identifiers and the number of packing station flag identifiers. An alarm message is issued when the packaging line at the target packaging station is in a tangled state.

2. The method for alarming tangled wires in packaging cables according to claim 1, characterized in that, During the training of the feature annotation model, obtaining the initial environmental image of the target packaging station further includes: Based on the initial environmental image of the target packaging station, the region of interest in the initial environmental image is determined; The regions outside the region of interest in the initial environment image are identified as regions of non-interest, and these regions of non-interest are filtered out.

3. The method for alarming tangled wires in packaging cables according to claim 1, characterized in that, After determining the packing line status of the target packing station based on the number of packing line identifiers and the number of packing station flag identifiers, the process further includes: If the packaging line at the target packaging station is in a tangled state, the feeding state of the packaging machine is determined to be abnormal. The abnormal feeding state of the packaging machine includes a missing packaging line and a tangled packaging line. The packaging machine is set at the target packaging station and is used to package products using the packaging line. The packaging station marker is set on the packaging machine or the target packaging station.

4. The method for alarming tangled wires in packaging lines according to claim 3, characterized in that, The following are considered abnormal feeding statuses for the packaging machine: The position of each packing line is determined based on the packing line markings. If at least two packing lines intersect, the abnormal feeding state is determined to be a packing line entanglement state. If the positions of any packing lines do not intersect, the abnormal feeding state is determined to be a packing line missing state.

5. The method for alarming tangled wires in packaging cables according to any one of claims 1-3, characterized in that, The target environment image is input into the packaging feature annotation model to obtain the packaging station identifier and packaging station flag identifier, including: The target environment image is input into the packaging feature annotation model to identify the packaging workstation marker features and packaging line features in the target environment image, and to obtain the packaging line annotation box and the packaging workstation marker annotation box. The packaging line marking frame is designated as the packaging line identifier, and the packaging station marker marking frame is designated as the packaging station marker identifier.

6. The method for alarming tangled wires in packaging cables according to any one of claims 1-3, characterized in that, Determining the packing line status of the target packing station based on the number of packing line identifiers and the number of packing station flag identifiers includes: Based on the packaging line identifier and the packaging station marker identifier, determine the number of packaging line identifiers and the number of packaging station marker identifiers; If the number of packaging line markers is equal to the number of packaging station markers, then the packaging line status of the target packaging station is determined to be normal. If the number of packaging line identifiers is less than the number of packaging station flag identifiers, then the packaging line status of the target packaging station is determined to be a disordered state.

7. The method for alarming tangled wires in packaging cables according to any one of claims 1-3, characterized in that, The alarm information issued based on the aforementioned disordered wiring status includes: Acquire multiple frames of environmental images of the target workstation and detect the packing line status of the target workstation in the multiple frames; When the packaging line status of the target station in the multiple frames is detected to be in a tangled state, the number of tangled frames in the multiple frames of the target station whose packaging line status is in a tangled state is determined. If the number of disordered frames is greater than or equal to a preset safe frame count threshold, an alarm message will be issued.

8. A tangled wire alarm device for packaging cables, characterized in that, include: The image acquisition module is used to acquire images of the target environment at the target packaging station; The identification module is used to input the target environment image into the packing feature annotation model to obtain packing line identifiers and packing station marker identifiers. The training method of the packing feature annotation model includes: acquiring an initial environment image of the target packing station, the initial environment image including red, green, and blue primary color channels; converting the red, green, and blue primary color channels, including swapping the brightness values ​​of the red and blue color channels to generate a converted environment image; determining the initial environment image and the converted environment image as sample images, and annotating the packing line features and packing station marker features in the sample images to generate a sample dataset; and training the initial annotation model based on the sample dataset. The initial annotation model after training is determined as the packing feature annotation model; wherein, the annotation of packing line features and packing station marker features in the sample image includes: rotating the packing line features and packing station marker features in the sample image using a rectangular bounding box, the rectangular bounding box including non-standard rectangular bounding boxes; correcting the non-standard rectangular bounding boxes to standard rectangular bounding boxes using a minimum bounding box; eliminating abnormal angle loss values ​​at the boundary based on the circular smoothing labeling method to obtain the angle prediction result of the packing feature annotation model; and detecting packing lines and packing station markers with rotation attributes within a preset angle range based on the angle prediction result to obtain packing line feature annotation boxes and packing station marker feature annotation boxes; The status determination module is used to determine the status of the packing line of the target packing station based on the number of packing line identifiers and the number of packing station flag identifiers. The alarm module is used to issue an alarm message when the packaging line at the target packaging station is in a tangled state.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the tangled wire alarm method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by the computer's processor, causes the computer to perform the packaging line tangled alarm method as described in any one of claims 1 to 7.

Images