Multi-pose image binding method and device, storage medium and electronic equipment

Abstract

The application discloses a multi-pose image binding method, a plurality of cameras are arranged at different positions of a target defect detection stage of a pipeline and connected to a same controller, the plurality of cameras synchronously capture products on the pipeline under control of the controller, the method further comprises: making a piece of the product enter a set abnormal state; acquiring captured images of all the cameras within a set time for all the cameras synchronously capturing the same product; performing image analysis on all the acquired captured images to identify a captured image of the product in the set abnormal state captured by each camera; calculating frame number differences between the identified captured images; taking the calculated frame number differences as interval product numbers between corresponding cameras; and performing multi-pose image binding based on the interval product numbers between the cameras. The application can correctly perform multi-pose image binding.

Description

Technical Field

[0001] This application relates to image technology, and in particular to a method, apparatus, storage medium and electronic device for multi-pose image binding of production line products. Background Technology

[0002] In industrial production, assembly lines are a common production mode. In some assembly line environments, a single product may exhibit different postures at different points in time. When a camera captures images of a single product in these different postures, it acquires multi-posture images of the product. These multi-posture images are beneficial for visual inspection of individual products, reducing missed inspections.

[0003] Taking a disposable glove production line as an example, disposable gloves mainly present three postures during the demolding stage of the production line: bottom, outer side, and inner side, such as... Figure 1 a , 1b As shown in Figure 1c, during the demolding stage, three industrial cameras are positioned at three different locations within the demolding section to capture images of the glove in three different poses. These three cameras are connected to a single photoelectric trigger device, which is activated once after each glove passes through the production line. Therefore, the three cameras capture images synchronously. The three poses of the same glove are captured by the industrial cameras at three different time points. When performing defect detection on the glove, it is necessary to bind the detection results of the three poses of the same glove (i.e., the images of the same glove captured by the three industrial cameras) together. The presence of a defect in the glove is confirmed by verifying the presence of defects in the three images. Similar multi-pose image binding requirements exist for other production line products.

[0004] Since several cameras on the production line take pictures of the same product in different poses simultaneously, binding multiple images of the same product can be transformed into determining the relative positions of several cameras. The relative positions can be calculated based on how many products (e.g., gloves) are spaced between two cameras.

[0005] To determine the number of products between two cameras when the production line stops, the relative position between the cameras can be determined by manually counting the products between them. However, due to the special nature of the production of certain products (such as gloves), the production line rarely stops and operates at a very high speed. It is impossible to determine the relative position by human eyes during production, and consequently, it is impossible to accurately bind multi-pose images. Summary of the Invention

[0006] This application provides a method, apparatus, storage medium, and electronic device for binding multi-pose images of production line products, which can correctly bind multi-pose images.

[0007] To achieve the above object, the application adopts the following technical scheme:

[0008] A binding method of multi-pose images, a plurality of cameras are arranged at different positions of a target defect detection stage of a production line and connected to a same controller, the plurality of cameras synchronously capture products on the production line under the control of the controller, and the method further comprises:

[0009] setting a product into a set abnormal state;

[0010] for all cameras synchronously capturing the same product, acquiring captured images of the all cameras within a set time; wherein the set time comprises a time during which the product is in the set abnormal state;

[0011] performing image analysis on the acquired all captured images to identify a captured image of the product in the set abnormal state captured by each camera;

[0012] calculating frame number differences between the identified captured images;

[0013] taking the calculated frame number differences as the number of interval products between the corresponding cameras;

[0014] performing binding of multi-pose images based on the number of interval products between the cameras.

[0015] Preferably, after the identification of the captured image of the product in the set abnormal state captured by each camera and before the calculation of the frame number differences, the method further comprises:

[0016] displaying the identified captured images, and after receiving an indication of correct image analysis, continuing to perform the operation of calculating the frame number differences.

[0017] Preferably, when the target defect detection stage of the production line is a demolding state detection stage of a glove production line, the set abnormal state of the product is an unmolded defect state;

[0018] setting the product into the set abnormal state is to separate the glove from a mold.

[0019] Preferably, the image analysis is BLOB analysis.

[0020] A binding device of multi-pose images, the device is connected to a plurality of cameras arranged at different positions of a target defect detection stage of a production line; the device comprises an interface unit, a state setting unit, a capturing control unit, an acquisition unit, an image analysis unit, an interval determination unit and an image binding unit;

[0021] the state setting unit is configured to set a product into a set abnormal state;

[0022] The photographing control unit is configured to control the plurality of cameras to synchronously photograph through the interface unit.

[0023] The acquisition unit is configured to acquire photographing images of all the cameras within a set time for all the cameras synchronously photographing the same product, wherein the set time comprises a time during which the product is in a set abnormal state.

[0024] The image analysis unit is configured to perform image analysis on all the acquired photographing images to identify a photographing image in which the product in a set abnormal state is photographed by each camera.

[0025] The interval determination unit is configured to calculate frame number differences between the identified photographing images, and take the calculated frame number differences as product numbers of intervals between corresponding cameras.

[0026] The image binding unit is configured to bind multi-pose images based on the product numbers of intervals between the cameras.

[0027] Preferably, a secondary confirmation unit is further included between the image analysis unit and the interval determination unit, configured to display the identified photographing images by the image analysis unit, and notify the interval determination unit to perform interval calculation after receiving an indication that the image analysis is correct.

[0028] Preferably, when the target defect detection stage in the production line is a demolding state detection stage in a glove production line, the set abnormal state of the product is an unmolded defect state.

[0029] In the state setting unit, the product is caused to enter the set abnormal state by causing the glove to be separated from the mold.

[0030] Preferably, the image analysis is BLOB analysis.

[0031] The application further provides a computer readable storage medium having computer instructions stored thereon, wherein the instructions are executed by a processor to implement the multi-pose image binding method of any one of the above.

[0032] The application further provides an electronic device comprising at least a computer readable storage medium and a processor.

[0033] The processor is configured to read the executable instructions from the computer readable storage medium and execute the instructions to implement the multi-pose image binding method of any one of the above.

[0034] From the above technical solution, in the present application, a product enters a set abnormal state; for a set time including the product in the set abnormal state, all camera shooting images in the set time are obtained for image analysis, and a shooting image of the product in the set abnormal state shot by each camera is identified; the frame number difference between the identified shooting images is calculated, and accordingly the number of products spaced between the corresponding cameras, i.e. the relative position between the cameras, can be determined; finally, the multi-pose image binding is performed based on the relative position between the cameras. In the above processing, the shooting image in the abnormal state is identified through image analysis, and since the shooting image in the abnormal state belongs to the same product, the frame number difference between the two shooting images in the abnormal state is the number of products spaced between the corresponding two cameras, so that the relative position between the cameras can be accurately determined without stopping the production line, and the multi-pose image binding can be accurately performed. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 a , Figure 1 b and Figure 1 c are examples of shooting images of three poses of PVC gloves in the demolding stage, respectively;

[0036] Figure 2 is a basic flow diagram of the multi-pose image binding method in the present application;

[0037] Figure 3a , 3b and 3c are examples of three camera shooting images in the demolding defect state;

[0038] Figure 4 is a specific flow diagram of the multi-pose image binding method in the specific embodiment of the present application;

[0039] Figure 5 is a basic structure diagram of the multi-pose image binding device in the present application;

[0040] Figure 6 is a basic structure diagram of the electronic device provided in the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical means and advantages of the present application more clear and explicit, the present application is further described in detail below with reference to the drawings.

[0042] In some abnormal states, the product morphology is quite different from the product morphology in normal state, and the probability of the product in such abnormal state is very small, and basically will not appear the same abnormal state in a relatively short period of time. Based on this, the basic idea of the present application is to identify the shooting images of the product in a specific abnormal state in the shooting images of each camera, and determine the relative position between different cameras based on the fact that the shooting images in abnormal state belong to the same product.

[0043] Figure 2 The basic flowchart of the multi-pose image binding method in the present application. In the method of the present application, N cameras are arranged at different positions of the target defect detection stage in a flow line and connected to the same controller. The N cameras synchronously shoot the product in the flow line under the control of the controller, and N is a positive integer greater than 1. As shown in Figure 2 , the binding method includes:

[0044] Step 201, set a product into a set abnormal state.

[0045] In order to distinguish between normal state and abnormal state, the product is set into a set abnormal state which is obviously different from the normal state. For example, in the demolding defect detection of the glove production flow line, the unmolded defect state can be used as the set abnormal state. The unmolded defect state is that the glove is combined with the mold together and has no demolding state, as shown in Figure 3a 、 3b and 3c. Therefore, making a glove in the flow line into an unmolded defect state can be to make the glove separate from the mold, thereby manufacturing an unmolded defect state.

[0046] Step 202, for all cameras shooting the same product synchronously, acquire the shooting images of all cameras within a set time.

[0047] The set time here includes the time when the product is in the set abnormal state, and the set time is less than a preset time threshold, which ensures that within the set time, no other product will be in the set abnormal state.

[0048] For the cameras that are synchronously shooting, the shooting image data of the cameras is acquired. In the image data, a frame number signal is contained, that is, the shooting sequence number of each shooting image in the corresponding camera. The multiple cameras that are synchronously shooting start streaming (that is, acquire the shooting image data) at the same time, the frame numbers all start from 1, and the multiple cameras are synchronously triggered to shoot once every time the production line moves to the position of one product, the frame number is then added by 1, and the corresponding camera reports the shooting image data to the controller once through the streaming callback event. Of course, the acquisition of the shooting image data can also be performed by the controller actively, but considering that the active acquisition of the controller will consume more CPU resources, therefore, in actual applications, preferably, the shooting image data can be sent in the manner that the cameras actively report.

[0049] In step 203, image analysis is performed on all the acquired shooting images, and one shooting image in which the product shot by each camera is in the set abnormal state is identified.

[0050] Since the set time includes the time during which the product is in the set abnormal state, the shooting images acquired include the images in which the product shot by each camera is in the set abnormal state. Since the product in the set abnormal state has a significant difference from the normal product, the shooting image in which the product is in the set abnormal state can be identified through image analysis.

[0051] For example, in the unmolded defect state of the glove, the shooting image is as shown in Figure 3a 、 3b and 3c, the shooting image of the glove in the normal mold removal stage is as shown in Figure 1 a 、 1b and 1c, obviously, the shooting image of the glove in the normal mold removal stage and the shooting image of the glove in the unmolded defect state have completely different ranges of the glove contour. Based on this, the shooting image in which the range of the glove contour is small can be determined to be in the unmolded defect state through image analysis (for example, BLOB analysis).

[0052] Of course, according to the difference between the products and the shooting images in the set abnormal state and the normal state, a suitable image analysis manner can be selected to identify the shooting image in the set abnormal state.

[0053] As described above, since the probability of the set abnormal state is very small, only one product is in the set abnormal state during the set time, and thus, in this step, one shooting image in the set abnormal state can be identified for each camera.

[0054] To further ensure the accuracy of the abnormal state recognition and multi-pose image binding, the identified individual captured images can be displayed, and after receiving a confirmation indication input by an operator, the step 204 is continued to be executed; if no confirmation indication is received, the step 201 is returned to select another product to enter the set abnormal state.

[0055] In step 204, the frame number difference between the identified individual captured images is calculated.

[0056] As described above, the frame number of the captured image is included in the captured image data, and through the step 203, for each camera, the captured image of the product in the set abnormal state photographed by the camera is identified, and the frame number of the identified captured image can be determined. N cameras are synchronously photographed, and N captured images can be identified, and the frame number difference between the N captured images can be calculated.

[0057] In step 205, the calculated frame number difference is taken as the number of products between the corresponding cameras.

[0058] First, analyze the relationship between the frame number difference of the same product photographed by different cameras and the relative position between the cameras. In two cameras X and Y, the camera X in the front position first photographs the image of the product M, at the same time, the camera Y also synchronously photographs, at this time, the frame numbers of the captured images of the two cameras are the same; then, the production line moves once, and the product moves, the camera X and the camera Y synchronously perform the next photographing, and the frame numbers of the captured images synchronously increase by 1, the production line continues to move, …, until the product M moves to another position behind the camera Y, the camera X and the camera Y synchronously perform image photographing, at this time, the product M is photographed from the camera X, it is assumed that the production line moves a times, the camera X and the camera Y also perform a times of photographing, and the frame numbers are increased by a, that is, the frame number difference between the captured images of the product M photographed by the camera X and the camera Y is a; therefore, the frame number difference between the captured images of the product M photographed by the camera X and the camera Y is equal to the number of times of the movement of the production line when the product M moves from the camera X to the camera Y, that is, the number of products between the camera X and the camera Y, which represents the relative position of the camera X and the camera Y.

[0059] In step 204, the frame number difference between the N images is calculated. One frame number difference corresponds to a pair of captured images A and B, the pair of captured images corresponds to two cameras X and Y, and A and B belong to the same product, then the frame number difference between the captured images A and B is equal to the number of products between the two cameras X and Y.

[0060] In this way, through the frame number difference obtained in the above step 204, the relative position of the corresponding two cameras can be determined.

[0061] Step 206, based on the product number between the cameras determined in step 205, the multi-pose image binding is performed.

[0062] The relative position relationship between the N cameras is determined through step 205, and based on the relative position relationship, the multi-pose image binding can be accurately performed. Specifically, the existing image binding method can be used, which will not be described here.

[0063] So far, Figure 2 The method flow shown ends.

[0064] From the above Figure 2 As can be seen from the method flow shown in the present application, the image analysis is used to identify the shooting image in the abnormal state, and since the shooting image in the abnormal state belongs to the same product, the frame number difference between the two shooting images in the abnormal state is the product number between the corresponding two cameras. Thus, the relative position between the cameras can be accurately determined without stopping the production line, and the multi-pose image binding can be accurately performed.

[0065] The specific flow of the multi-pose image binding method in the present application will be described below through a specific embodiment. Figure 4 The specific flow of the multi-pose image binding method in the present application is shown in the specific embodiment. In the specific embodiment, it is assumed that defect detection is performed at the demolding stage of the production line for disposable gloves, three cameras are arranged at three different positions at the demolding stage to shoot images of the bottom, the outer side and the inner side, and the three cameras are connected to a controller to shoot synchronously. Each camera actively reports the shooting image data to the controller through a stream callback event. As shown in Figure 4 The method flow specifically includes:

[0066] Step 401, a glove is separated from a mold;

[0067] The state of a glove separated from a mold is equivalent to the glove in the unmolded defect state, so that the glove is in the unmolded defect state through the operation of step 401;

[0068] Step 402, shooting images obtained by each camera within a set time are acquired;

[0069] The set time includes the time when the glove is separated from the mold, and should not be too large.

[0070] Step 403, BLOB analysis is performed on the acquired shooting images to identify the shooting images of the glove in the unmolded defect state.

[0071] Through image BLOB analysis, the unmolded defect glove images of the bottom, the outer side and the inner side are screened out;

[0072] Step 404, display the recognized glove image on the interface;

[0073] Step 403, display the glove image screened out by the step 402 on the interface, so as to facilitate the operator to make a secondary confirmation;

[0074] Step 405, judge whether the confirmation instruction is received, if yes, continue to execute step 406, otherwise return to step 401;

[0075] When the operator confirms that the screened out shooting image is indeed in the set abnormal state (the un-molded defect state in the embodiment), the confirmation instruction is sent, indicating that the image analysis is correct; when the operator thinks that the screened out shooting image is not all in the set abnormal state, the confirmation instruction is not sent, indicating that the image analysis has a problem, and then the step 401 is returned to select another glove to enter the un-molded defect state;

[0076] Step 406, calculate the frame number difference between the recognized shooting images as the product number of the interval between the corresponding cameras;

[0077] For example, it is assumed that the actual frame numbers of the three screened out images are shown in Table 1,

[0078] bottom end outer side inner side 46675 46703 46705

[0079] Table 1

[0080] Therefore, the relative positions of the three cameras can be calculated as shown in Table 2.

[0081] bottom end outer side inner side 0 28 30

[0082] Table 2

[0083] Step 407, according to the product number of the interval between the cameras, the multi-pose image binding is performed.

[0084] So far, Figure 4 The method flow shown in the figure ends.

[0085] As can be seen from the specific implementation of the above embodiment, the multi-pose image binding method provided by the application can quickly calculate the binding position relationship of the multi-pose image, avoiding the time and manpower of a large number of manual searching and association of pictures; at the same time, the binding process can be completed without stopping the line.

[0086] The application also provides a multi-pose image binding device, which can be used to implement the multi-pose image binding method described above. The device is connected with a plurality of cameras arranged at different positions of a target defect detection stage of a flow line, and specifically, the device can be located in a controller. Figure 5 The basic structure diagram of the multi-pose image binding device provided by the application is shown in the figure.Figure 5 As shown, the device comprises an interface unit, a shooting control unit, a state setting unit, an acquisition unit, an image analysis unit, an interval determination unit and an image binding unit.

[0087] The state setting unit is configured to set one of the products into a set abnormal state.

[0088] The shooting control unit is configured to control the plurality of cameras to synchronously shoot through the interface unit.

[0089] The acquisition unit is configured to acquire shooting images of all the cameras within a set time for all the cameras synchronously shooting the same product; the set time comprises a time during which one product is in the set abnormal state.

[0090] The image analysis unit is configured to perform image analysis on the acquired shooting images, and identify one shooting image of the product in the set abnormal state captured by each camera.

[0091] The interval determination unit is configured to calculate frame number differences between the identified shooting images, and take the calculated frame number differences as the number of products between the corresponding cameras.

[0092] The image binding unit is configured to bind the multi-pose images based on the number of products between the cameras.

[0093] Through the processing of the device, the relative positions between the cameras can be accurately determined without stopping the pipeline, and the binding accuracy of the multi-pose images is improved.

[0094] In addition, optionally, a secondary confirmation unit can be further included between the image analysis unit and the interval determination unit, configured to display the identified shooting images of the image analysis unit, and notify the interval determination unit to perform interval calculation after receiving an indication that the image analysis is correct.

[0095] Optionally, when the target defect detection stage of the pipeline is a demolding state detection stage of the glove production pipeline, the set abnormal state of the product is an unmolded defect state.

[0096] In the state setting unit, making the product into the set abnormal state comprises making the glove separate from the mold.

[0097] Optionally, the image analysis can be BLOB analysis.

[0098] The application further provides a computer readable storage medium storing instructions executable by a processor to perform the steps of the multi-pose image binding method.

[0099] According to the embodiments disclosed in the application, the computer readable storage medium can be a non-volatile computer readable storage medium, for example, can include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, but not to limit the scope of protection of the application. In the embodiments disclosed in the application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus.

[0100] Figure 6 The application further provides an electronic device. As shown in Figure 6 , which shows a structural schematic diagram of an electronic device related to the embodiments of the application, specifically:

[0101] The electronic device can include a processor 601 with one or more processing cores, a memory 602 with one or more computer readable storage media, and a computer program stored on the memory and executable on the processor. When executing the program of the memory 602, the multi-pose image binding method can be implemented.

[0102] Specifically, in practical applications, the electronic device can further include a power supply 603, an input and output unit 604 and the like. Those skilled in the art can understand that Figure 6 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and can include more or fewer components than those shown, or combine certain components, or different component arrangements. Among them:

[0103] The processor 601 is the control center of the electronic device, which connects all parts of the electronic device through various interfaces and lines, executes the software programs and / or modules stored in the memory 602, and calls the data stored in the memory 602, executes various functions of the server and processes data, thereby overall monitoring the electronic device.

[0104] The memory 602 can be used to store software programs and modules, i.e., the computer readable storage medium described above. The processor 601 performs various functions and data processing by running the software programs and modules stored in the memory 602. The memory 602 can mainly include a program storage area and a data storage area, wherein the program storage area can store operating systems, application programs required by at least one function, etc.; and the data storage area can store data created according to the use of the server, etc. In addition, the memory 602 can include a high-speed random access memory, and can also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 602 can also include a memory controller to provide the processor 601 with access to the memory 602.

[0105] The electronic device also includes a power supply 603 for supplying power to various components, which can be logically connected to the processor 601 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 503 can also include one or more direct current or alternating current power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, etc.

[0106] The electronic device can also include an input / output unit 604, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. The input / output unit 604 can also be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof.

[0107] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for binding multi-pose images, characterized in that, In a method for detecting defects on a production line, multiple cameras are installed at different locations and connected to the same controller. These cameras synchronously capture images of products on the production line under the control of the controller. The method further includes: To cause one of the products to enter a set abnormal state; For all cameras simultaneously capturing images of the same product, acquire images captured by all cameras within a set time period; wherein, the set time period includes the time during which one of the products is in a set abnormal state; Image analysis is performed on all acquired images to identify the image captured by each camera that shows the product in a set abnormal state. Calculate the frame number difference between the identified images; The calculated frame number difference is used as the number of products in the corresponding camera interval; Binding of multi-pose images is performed based on the number of products between the cameras.

2. The method according to claim 1, characterized in that, After identifying an image of the product captured by each camera in a predefined abnormal state and before calculating the frame number difference, the method further includes: The identified images are displayed, and after receiving a correct image analysis indication, the operation of calculating the frame number difference continues.

3. The method according to claim 2, characterized in that, When the target defect detection stage on the production line is the demolding state detection stage on the glove production line, the set abnormal state of the product is the undemolded defect state. The product is set to enter an abnormal state by causing the gloves to detach from the mold.

4. The method according to claim 1, characterized in that, The image analysis is BLOB analysis.

5. A binding device for multi-pose images, characterized in that, The device is connected to multiple cameras installed at different locations in the target defect detection stage of a production line; the device includes: an interface unit, a status setting unit, an image capture control unit, an acquisition unit, an image analysis unit, an interval determination unit, and an image binding unit; The status setting unit is used to set a product on the production line into a set abnormal state. The shooting control unit is used to control the multiple cameras to simultaneously shoot the product through the interface unit; The acquisition unit is used to acquire images captured by all cameras simultaneously shooting the product within a set time period; wherein, the set time period includes the time during which the product is in a set abnormal state; The image analysis unit is used to perform image analysis on all acquired images and identify one image captured by each camera in which the product is in a set abnormal state. The interval determination unit is used to calculate the frame number difference between each captured image and to use the calculated frame number difference as the number of products corresponding to the interval between cameras. The image binding unit is used to bind multi-pose images based on the number of products between the cameras.

6. The apparatus according to claim 5, characterized in that, A secondary confirmation unit is further included between the image analysis unit and the interval determination unit. This unit is used to display each captured image identified by the image analysis unit and, upon receiving a correct image analysis indication, to notify the interval determination unit to perform interval calculation.

7. The apparatus according to claim 6, characterized in that, When the target defect detection stage on the production line is the demolding state detection stage on the glove production line, the set abnormal state of the product is the undemolded defect state. In the state setting unit, the way to put the product into a set abnormal state is to remove the glove from the mold.

8. The apparatus according to claim 5, characterized in that, The image analysis is BLOB analysis.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instruction is executed by the processor, it can implement the multi-pose image binding method according to any one of claims 1 to 4.

10. An electronic device, characterized in that, The electronic device includes at least a computer-readable storage medium and a processor; The processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the multi-pose image binding method according to any one of claims 1 to 4.

Images