Assembly line product counting method, storage medium and device

Abstract

The invention relates to the technical field of visual inspection, and discloses an assembly line product counting method, a storage medium and a device, and the method comprises the following steps: continuously photographing products on an assembly line; identifying the position of each product on the two continuous photos, and calculating the actual moving distance of each product according to the position of each product in the two continuous photos; calculating the theoretical movement distance of the product according to the linear speed of the assembly line and the photographing interval; calculating the safety margin according to the equipment calibration data, the vibration amplitude and the pipeline speed; if the actual moving distance of the product is larger than or equal to the sum of the theoretical moving distance and the safety margin, adding one to the product count value. According to the method, the threshold value for judging whether the counting is repeated or not is set according to the equipment calibration data, so that the method is more suitable for individual difference of each piece of equipment compared with a method for setting the threshold value depending on artificial experience, and the counting accuracy is improved. Meanwhile, the threshold value dynamically changes along with the assembly line speed, and the influence of assembly line speed fluctuation can be eliminated.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to a method, storage medium and device for counting products on an assembly line. Background Art

[0002] Assembly line equipment is an essential component of automated production lines for various products. In some assembly lines, it's often necessary to count the number of products. Existing product counting solutions often rely on photoelectric sensors or machine vision to count products on the assembly line. In machine vision-based product counting, due to the similar appearance of products on the assembly line, duplicate counting is prone to occur. To address this technical issue, existing visual counting solutions often take consecutive photos of products on the assembly line, compare the actual travel distance of the product in two consecutive photos with the theoretical travel distance of the product, and count products where the difference between the actual and theoretical travel distances exceeds a fixed threshold as newly entered the camera's field of view, thereby avoiding duplicate counting of the same product. Existing thresholds for determining whether a product is being counted are often set based on manual experience. However, in practice, the speed of the assembly line's conveyor mechanism fluctuates, and interference such as equipment vibration occurs during operation. This fixed threshold, set based on manual experience, can still lead to errors in counting, resulting in missed or duplicate counts. Summary of the Invention

[0003] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a method for counting products on an assembly line, which improves the accuracy of counting products on the assembly line through machine vision by setting a dynamic safety margin based on equipment calibration data, vibration amplitude and assembly line speed.

[0004] To solve the above problems, the technical solution adopted by the present invention is as follows: a method for counting products on an assembly line, comprising the following steps: Continuously take photos of products on the assembly line; Identify the position of each product in two consecutive photos, and calculate the actual movement distance of each product based on the position of each product in the two consecutive photos; Calculate the theoretical moving distance of the product based on the line speed and the shooting interval of the assembly line; Calculate safety margin based on equipment calibration data, vibration amplitude and maximum line speed; If the actual moving distance of the product is greater than or equal to the theoretical moving distance + safety margin, it is determined that a new product has entered the camera's field of view and the product count value is increased by one.

[0005] Compared with the existing technology, the beneficial effect of the present invention is that: this counting method sets a safety margin according to the equipment calibration data and adjusts the safety margin according to the speed and vibration amplitude of the assembly line, so as to more accurately determine whether the products in the camera's field of view are products to be counted repeatedly, and eliminate the influence of speed fluctuations and equipment vibrations during equipment operation, thereby improving the accuracy of product counting.

[0006] In the above-mentioned method for counting products on an assembly line, in the step of calculating the safety margin based on the equipment calibration data and the assembly line speed, the calculation formula for the safety margin S is as follows:

[0007] Where k is the safety factor, is the maximum speed of the pipeline, is the standard deviation of mechanical vibration, To compensate for vibration disturbance, Compensation for speed fluctuations.

[0008] The above-mentioned pipeline product counting method, the vibration interference compensation ,in is the vibration sensitivity coefficient, is the real-time vibration amplitude of the equipment.

[0009] The above-mentioned pipeline product counting method, the speed fluctuation compensation ,in is the velocity fluctuation coefficient, is the current speed of the pipeline, is the average speed of the pipeline.

[0010] The above-mentioned method for counting products on an assembly line, after the step of calculating the safety margin based on the equipment calibration data and the maximum speed of the assembly line, further comprises: The safety margin is corrected according to the counting misjudgment rate.

[0011] In the above-mentioned method for counting products on an assembly line, the step of correcting the safety margin according to the counting error rate includes: Calculate the misjudgment rate based on the product count value and the actual product quantity; If the false positive rate is positive and greater than the false positive upper threshold, the safety margin is increased by a certain value; If the false positive rate is negative and less than the false positive lower limit threshold, the safety margin is reduced by a certain value.

[0012] In the above-mentioned pipeline product counting method, in the steps of identifying the position of each product on the two photos, calculating the position of each product in the two photos, and calculating the actual movement distance of each product, the position of each product on the photo is identified by a trained residual network model.

[0013] A storage medium stores a computer program, wherein the computer program is configured to implement the above-mentioned pipeline product counting method when running.

[0014] A device for counting products on an assembly line comprises a processor and a memory, wherein the processor is electrically connected to the memory and the processor can implement the above-mentioned method for counting products on the assembly line by calling and executing a computer program in the memory.

[0015] A production line device comprises the above-mentioned production line product counting device, a camera, an amplitude sensor, a speed sensor and a conveying mechanism, wherein the camera is arranged above the conveying mechanism and is used to take pictures of the products of the conveying mechanism; the amplitude sensor and the speed sensor are both arranged on the conveying mechanism; the amplitude sensor is used to detect the real-time vibration amplitude of the conveying mechanism; the speed sensor is used to detect the real-time speed of the conveying mechanism; the camera, the amplitude sensor and the speed sensor are all electrically connected to the production line product counting device.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure is a flow chart of a method for counting products on an assembly line according to an embodiment of the present invention.

[0018] Figure 2 This is a principle block diagram of the pipeline equipment according to an embodiment of the present invention.

[0019] Description of Figure Numbers: 100 conveying mechanism, 200 camera, 300 speed sensor, 400 amplitude sensor, 500 assembly line product counting device. DETAILED DESCRIPTION

[0020] The embodiments of the present invention are described in detail below. Figure 1 , an embodiment of the present invention provides a method for counting products on an assembly line, comprising the following steps: Continuously take photos of products on the assembly line; Identify the position of each product in two consecutive photos, and calculate the actual movement distance of each product based on the position of each product in the two consecutive photos; Calculate the theoretical moving distance of the product based on the line speed and the shooting interval of the assembly line; Calculate safety margins based on equipment calibration data, vibration amplitude, and line speed; If the actual moving distance of the product is greater than or equal to the theoretical moving distance + the safety margin, it is determined that a new product has entered the field of view of the camera 200, and the product count value is increased by one.

[0021] This method calculates a safety margin based on equipment calibration data and assembly line speed. This allows the threshold for determining whether a product is a repeat count to be set based on the specific equipment when counting products using machine vision. This makes the threshold more consistent with the actual assembly line equipment, thereby improving the accuracy of repeat count determinations compared to thresholds set based on manual experience. Furthermore, this method incorporates a safety margin that is dynamic, related to the line speed and vibration amplitude of the assembly line. This allows the threshold for determining whether a product is a repeat count to be adjusted in real time as the assembly line speed fluctuates, thus avoiding potential misjudgments caused by fluctuations in assembly line speed and equipment vibration.

[0022] In this embodiment, for the convenience of calculation, the actual movement distance of the product, the theoretical movement distance of the product, and the safety margin are all in pixels. The actual movement distance of the product is obtained by the corresponding coordinate difference of the corner or geometric center of the rectangular frame of each product in two consecutive photos identified by the neural network. The theoretical movement distance of the product is , where V is the speed of the pipeline, and the set speed of the pipeline is generally used. is the photo interval, and P is the pixel resolution.

[0023] In this embodiment, specifically, the calculation formula of the safety margin S is as follows: (1) Where k is the safety factor, usually 1.2-1.5, is the maximum speed of the pipeline, is the standard deviation of mechanical vibration in pixels. To compensate for vibration interference, it is calculated by the following formula: (2) in is the vibration sensitivity coefficient of the equipment, The real-time vibration amplitude of the device. To compensate for speed fluctuations, it is calculated based on the real-time line speed of the assembly line, as shown in the following formula: (3) in is the speed fluctuation coefficient of the equipment, is the current speed of the pipeline, is the average speed of the assembly line, which is generally the average speed in the recent period. and velocity fluctuation coefficient All are obtained by calibrating the equipment in advance.

[0024] In some embodiments, in order to further improve the accuracy of repeated counting determination, the calculated safety margin is corrected in real time according to the misjudgment rate each time a determination is made.

[0025] In practice, the false positive rate can be calculated as (number of products counted by the device - actual number of products) / actual number of products. When the false positive rate is positive, it means that the device counted more products than the actual number of products. The threshold for determining whether duplicate counting is too low, and the safety margin should be increased to raise the threshold. When the false positive rate is negative, it means that the device counted fewer products than the actual number of products. The threshold for determining whether duplicate counting is too high, and the safety margin should be reduced to lower the threshold. In this embodiment, if the false positive rate is positive and greater than the false positive upper threshold, such as +0.5%, then the safety margin needs to be increased by 10 pixels, i.e. , is the revised new safety margin, is the safety margin calculated according to the above formula (1); if the misjudgment rate is negative and less than the misjudgment lower limit threshold, such as -0.5%, the safety margin needs to be reduced by 5 pixels, that is, -5.

[0026] In this example, to further reduce the probability of misidentification and improve the accuracy of calculating the product's actual travel distance, a residual network model (ResNet) is used to identify and locate products in continuous photos, avoiding the interference of traditional template matching, which is susceptible to lighting and product deformation. The residual network model is trained on a sample set consisting of more than 500 dynamic product images collected in real life, annotated with rectangular boxes of the product's orientation using labelImg. The model is trained using the Pytorch framework, using a Smooth L1 Loss optimization loss function, and training is performed until the validation set accuracy exceeds 99.9%. After model training, the model is exported in ONNX format and integrated into the Halcon vision platform.

[0027] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned pipeline product counting method can be implemented.

[0028] In some possible implementations, various aspects of the pipeline product counting method provided by the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the pipeline product counting method according to various exemplary embodiments of the present application described above in this specification.

[0029] Based on the same inventive concept, an embodiment of the present invention also provides a counting device for implementing the above-mentioned pipeline product counting method, including a processor and a memory, the memory is electrically connected to the processor, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned pipeline product counting method.

[0030] In one possible design, the processor may include one or more processing units, and the processor and memory may be implemented on the same chip or separately on independent chips. The processor may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the pipeline product counting method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0031] As a non-volatile computer-readable storage medium, memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules.Memory can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic storage, disk, optical disk, etc. Memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application can also be a circuit or other arbitrarily capable of implementing a storage function, for storing program instructions and / or data.

[0032] By designing and programming a processor, the code corresponding to the pipeline product counting method described in the aforementioned embodiment can be embedded in the chip, thereby enabling the chip to execute the steps of the pipeline product counting method described in the embodiment of the present invention when running. Designing and programming a processor is well known to those skilled in the art and will not be further described here.

[0033] Based on the same inventive concept, an embodiment of the present invention further provides an assembly line device, comprising the above-mentioned assembly line product counting device 500, a camera 200, an amplitude sensor 400, a speed sensor 300, and a conveying mechanism 100. The conveying mechanism 100 can be a conveyor belt or a roller conveyor. The speed sensor 300 generally adopts an encoder, which is installed on a drive shaft of the conveyor belt mechanism and connected to the drive shaft to obtain the real-time linear speed of the conveyor mechanism 100, thereby obtaining the real-time linear speed of the assembly line. The amplitude sensor 400 is provided on the conveyor belt mechanism to monitor the amplitude of the conveyor belt mechanism. The camera 200 is provided above the conveyor mechanism 100 to continuously take pictures of the products on the conveyor mechanism 100. The camera 200, amplitude sensor 400, and speed sensor 300 are all electrically connected to the assembly line product counting device 500. This transmits photos, the device's real-time vibration amplitude, and speed to the assembly line product counting device 500, enabling it to calculate the actual travel distance and safety margin of the products. By implementing the aforementioned assembly line product counting method, accurate product counting on the assembly line is achieved. Actual testing has shown that the product counting error of this assembly line device can be controlled to within ±0.5%.

[0034] It should be noted that in the description of the present invention, "several" means one or more, "more" means two or more, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. If there are descriptions of "first," "second," and so on, these are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0035] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0036] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0037] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0038] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0039] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A method for counting products on an assembly line, characterized in that: The steps include: Continuously take photos of products on the assembly line; Identify the position of each product in two consecutive photos, and calculate the actual movement distance of each product based on the position of each product in the two consecutive photos; Calculate the theoretical moving distance of the product based on the line speed and the shooting interval of the assembly line; Calculate safety margins based on equipment calibration data, vibration amplitude, and line speed; If the actual moving distance of the product is greater than or equal to the theoretical moving distance + the safety margin, it is determined that a new product has entered the field of view of the camera (200), and the product count value is increased by one.

2. The method for counting products on an assembly line according to claim 1, characterized in that: In the step of calculating the safety margin based on the equipment calibration data and the assembly line speed, the calculation formula of the safety margin S is as follows: Where k is the safety factor, is the maximum speed of the pipeline, is the standard deviation of mechanical vibration, To compensate for vibration disturbance, Compensation for speed fluctuations.

3. The method for counting products on an assembly line according to claim 2, characterized in that: Vibration disturbance compensation ,in is the vibration sensitivity coefficient, is the real-time vibration amplitude of the equipment.

4. The method for counting products on an assembly line according to claim 2, wherein: The speed fluctuation compensation ,in is the velocity fluctuation coefficient, is the current speed of the pipeline, is the average speed of the pipeline.

5. The method for counting products on an assembly line according to claim 1, wherein: After the step of calculating the safety margin based on the equipment calibration data and the maximum speed of the assembly line, the method further includes: The safety margin is corrected according to the counting misjudgment rate.

6. The method for counting products on an assembly line according to claim 5, characterized in that: The step of correcting the safety margin according to the counting misjudgment rate includes: Calculate the misjudgment rate based on the product count value and the actual product quantity; If the false positive rate is positive and greater than the false positive upper threshold, the safety margin is increased by a certain value; If the false positive rate is negative and less than the false positive lower limit threshold, the safety margin is reduced by a certain value.

7. The method for counting products on an assembly line according to claim 1, wherein: In the steps of identifying the position of each product on the two photos, calculating the position of each product in the two photos, and calculating the actual movement distance of each product, the position of each product on the photos is identified by a trained residual network model.

8. A storage medium storing a computer program, characterized in that: The computer program is configured to implement the assembly line product counting method according to any one of claims 1 to 7 when running.

9. A product counting device for an assembly line, characterized in that: It includes a processor and a memory, the processor is electrically connected to the memory, and the processor can implement the assembly line product counting method according to any one of claims 1 to 7 by calling and executing the computer program in the memory.

10. A production line device, characterized in that: The invention comprises an assembly line product counting device (500) according to claim 9, a camera (200), an amplitude sensor (400), a speed sensor (300) and a conveying mechanism (100), wherein the camera (200) is arranged above the conveying mechanism (100), the camera (200) is used to take pictures of products of the conveying mechanism (100), the amplitude sensor (400) and the speed sensor (300) are both arranged on the conveying mechanism (100), the amplitude sensor (400) is used to detect the real-time vibration amplitude of the conveying mechanism (100), the speed sensor (300) is used to detect the real-time speed of the conveying mechanism (100), and the camera (200), the amplitude sensor (400) and the speed sensor (300) are all electrically connected to the assembly line product counting device (500).

Images