Method, device and system for measuring the length, width and height of an object in an industrial assembly line

Abstract

An embodiment of the present invention discloses a method, device and system for measuring the length, width and height of an object in an industrial assembly line. Ultrasonic ranging is performed on the object on the conveyor belt of the assembly line to obtain the height information of the object; an image of the object is collected, and the object image is segmented according to the obtained object image information by using the GrabCut algorithm, and the length and width of the object in the image are obtained; the object is modeled by using the principle of similar triangles to obtain the length and width information of the object. It can realize the intelligent detection of the size of storage packages, with high detection efficiency and good accuracy.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of object detection, and particularly to a method, device and system for measuring the length, width and height of an object in an industrial assembly line. Background Art

[0002] In the existing industrial warehousing system, packages are usually classified by rough classification, which mainly includes large packages, medium packages and small packages. In traditional technologies, the classification and judgment of the size of packages are relatively rough and require manual intervention, so there are problems of low efficiency and low accuracy. If we want to make the warehousing more intelligent and achieve the goal of unmanned warehousing, then accurately measuring the length, width and height of packages is an inevitable way. Summary of the Invention

[0003] Therefore, the embodiments of the present invention provide a method, device and system for measuring the length, width and height of an object in an industrial assembly line to solve the problems in the existing technology that the classification and judgment of the size of packages are relatively rough, require manual intervention, have low efficiency and low accuracy.

[0004] To achieve the above object, the embodiments of the present invention provide the following technical solutions:

[0005] According to the first aspect of the embodiments of the present invention, a method for measuring the length, width and height of an object in an industrial assembly line is proposed. The method includes:

[0006] Performing ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object;

[0007] Performing image acquisition on the object, segmenting the object image by using the GrabCut algorithm according to the obtained object image information, and obtaining the length and width of the object in the image;

[0008] Using the principle of similar triangles to model the object to obtain the length and width information of the object.

[0009] Further, performing ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object specifically includes:

[0010] Real-time measuring the distance H from the ultrasonic module deployed above the conveyor belt to the conveyor belt;

[0011] When the object is conveyed under the ultrasonic module, measuring the distance h from the ultrasonic wave to the upper surface of the object;

[0012] The height of the object is H - h.

[0013] Further, an image of the object is collected, and the GrabCut algorithm is used to segment the object image based on the obtained object image information, and the length and width of the object in the image are obtained. Specifically, it includes:

[0014] The object is imaged by an image acquisition module disposed above the conveyor belt. The image acquisition module and the ultrasonic module are arranged on the same horizontal line.

[0015] A rectangular area containing the object is set as the ROI area for the collected object image, and the area outside the rectangular area is automatically set as the background.

[0016] A mask image of the same size as the object image is obtained, filled with pixel 0, and a foreground and background model is created.

[0017] Based on the GrabCut algorithm function, the object segmentation image is obtained by image segmentation, and the four vertex coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) of the object in the image coordinate system are obtained. The length box_pix_l and width box_pix_w of the pixel values of the object image are respectively:

[0018]

[0019]

[0020] Further, using the principle of similar triangles, the object is modeled to obtain the length and width information of the object. Specifically, it includes:

[0021] The pixel values camera_pix_l and camera_pix_w of the camera are obtained, and the focal point coordinates (f_x, f_y) and the center point coordinates (x0, y0) of the camera are obtained according to the internal parameter matrix of the camera.

[0022] The maximum length bg_l and width bg_w projected by the camera on the conveyor belt are respectively:

[0023] bg_w = (bg_dis * x0) / f_x

[0024] bg_l = (bg_dis * y0) / f_y;

[0025] Then the length each_pix_l and width each_pix_w of each pixel value corresponding to the maximum length and width projected by the camera on the conveyor belt are respectively:

[0026] each_pix_w = bg_w / camera_pix_w

[0027] each_pix_l = bg_l / camera_pix_l;

[0028] Then the length box_bg_l and width box_bg_w of the object's projection on the conveyor belt are respectively:

[0029] box_bg_w = box_pix_w * each_pix_w

[0030] box_bg_l = box_pix_l * each_pix_l;

[0031] Then the length boxself_l and width boxself_w of the object itself are respectively:

[0032] boxself_w = (box_bg_w * (bg_dis * boxself_h)) / bg_dis

[0033] boxself_l = (box_bg_l * (bg_dis * boxself_h)) / bg_dis

[0034] Wherein, bg_dis is the distance H between the ultrasonic module's real-time measurement and the conveyor belt, and boxself_h is the distance h between the ultrasonic measurement and the upper surface of the object.

[0035] According to the second aspect of the embodiments of the present invention, a device for measuring the length, width, and height of an object in an industrial assembly line is proposed. The device includes:

[0036] An ultrasonic ranging module for performing ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object;

[0037] An image acquisition module for acquiring an image of the object;

[0038] A processing module for performing image segmentation on the acquired object image information using the GrabCut algorithm and obtaining the length and width of the object in the image;

[0039] Modeling the object using the principle of similar triangles to obtain the length and width information of the object.

[0040] According to the third aspect of the embodiments of the present invention, a system for measuring the length, width, and height of an object in an industrial assembly line is proposed. The system includes: a processor and a memory;

[0041] The memory is used to store one or more program instructions;

[0042] The processor is used to run one or more program instructions to execute the method described in any one of the above.

[0043] According to a fourth aspect of an embodiment of the present invention, a computer storage medium is provided. The computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a system for measuring the length, width, and height of an object in an industrial pipeline to perform the method described in any one of the above.

[0044] The embodiments of the present invention have the following advantages:

[0045] A method, device, and system for measuring the length, width, and height of an object in an industrial pipeline proposed in the embodiments of the present invention perform ultrasonic ranging on an object on a conveyor belt of the pipeline to obtain the height information of the object; perform image acquisition on the object, segment the object image using the GrabCut algorithm according to the obtained object image information, and obtain the length and width of the object in the image; use the principle of similar triangles to model the object to obtain the length and width information of the object. It can realize intelligent detection of the size of storage packages, with high detection efficiency and good accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can also be obtained according to the provided drawings.

[0047] Figure 1 It is a schematic flowchart of a method for measuring the length, width, and height of an object in an industrial pipeline provided in Embodiment 1 of the present invention;

[0048] Figure 2 It is an object image collected in the method for measuring the length, width, and height of an object in an industrial pipeline provided in Embodiment 1 of the present invention;

[0049] Figure 3 It is a mask image with a black background in the method for measuring the length, width, and height of an object in an industrial pipeline provided in Embodiment 1 of the present invention;

[0050] Figure 4 It is an object segmentation image obtained in the method for measuring the length, width, and height of an object in an industrial pipeline provided in Embodiment 1 of the present invention;

[0051] Figure 5 It is a schematic structural diagram of a device for measuring the length, width, and height of an object in an industrial pipeline provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The following specific embodiments illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0053] Embodiment 1

[0054] As Figure 1 shown, this embodiment is based on the Raspberry Pi platform and proposes a method for measuring the length, width, and height of an object in an industrial assembly line. The method includes:

[0055] S110. Perform ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object.

[0056] Step S110 specifically includes:

[0057] Real-time measure the distance H from the ultrasonic module deployed above the conveyor belt to the conveyor belt;

[0058] When the object is conveyed under the ultrasonic module, measure the distance h from the ultrasonic module to the upper surface of the object through ultrasonic measurement;

[0059] The height of the object is H - h.

[0060] S120. Perform image acquisition on the object, segment the object image using the GrabCut algorithm according to the obtained object image information, and obtain the length and width of the object in the image.

[0061] The implementation steps of the GrabCut algorithm are as follows: 1. Define one or more rectangles containing the object in the picture. 2. The area outside the rectangle is automatically considered as the background. 3. For the user-defined rectangular area, the data in the background can be used to distinguish the foreground and background areas inside it. 4. Use the Gaussian Mixture Model (GMM) to model the background and foreground, and mark the undefined pixels as possible foreground or background. 5. Each pixel in the image is regarded as connected to the surrounding pixels through virtual edges, and each edge has a probability of belonging to the foreground or background, which is based on its color similarity with the surrounding pixels. 6. Each pixel (i.e., a node in the algorithm) will be connected to a foreground or background node. 7. After the nodes are connected (possibly connected to the background or foreground), if the edges between the nodes belong to different terminals (i.e., one node belongs to the foreground and the other node belongs to the background), then the edges between them will be cut off, which can segment each part of the image.

[0062] Step S120 specifically includes:

[0063] The object is imaged by an image acquisition module disposed above the conveyor belt. The image acquisition module and the ultrasonic module are arranged on the same horizontal line, that is, the ultrasonic module is deployed above the conveyor belt and parallel to the monocular camera. When an object approaches, the camera captures an image of the object on the conveyor belt;

[0064] For the captured object image (as Figure 2 shown), a rectangular area containing the object is set as the ROI area, and the area outside the rectangular area is automatically set as the background;

[0065] A mask image of the same size as the object image is obtained and filled with pixel 0 (as Figure 3 shown) to create a foreground and background model;

[0066] Based on the GrabCut algorithm function, image segmentation is performed to obtain an object segmentation image (as Figure 4 shown), and the four vertex coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) of the object in the image coordinate system are obtained. The length box_pix_l and width box_pix_w of the object's image pixel values are respectively:

[0067]

[0068]

[0069] S130. Using the principle of similar triangles, model the object to obtain the length and width information of the object.

[0070] Step S130 specifically includes:

[0071] Using the principle of similar triangles, model the object to obtain the length and width information of the object, specifically including:

[0072] Obtain the pixel values camera_pix_l and camera_pix_w of the camera, and obtain the focal point coordinates (f_x, f_y) and the center point coordinates (x0, y0) of the camera according to the internal parameter matrix of the camera;

[0073] The maximum length bg_l and width bg_w projected by the camera on the conveyor belt are respectively:

[0074] bg_w = (bg_dis * x0) / f_x

[0075] bg_l = (bg_dis * y0) / f_y;

[0076] Then the length each_pix_l and width each_pix_w corresponding to each pixel value of the maximum length and width projected by the camera on the conveyor belt are respectively:

[0077] each_pix_w = bg_w / camera_pix_w

[0078] each_pix_l = bg_l / camera_pix_l;

[0079] Then the length box_bg_l and width box_bg_w of the object projected on the conveyor belt are respectively:

[0080] box_bg_w = box_pix_w * each_pix_w

[0081] box_bg_l = box_pix_l * each_pix_l;

[0082] Then the length boxself_l and width boxself_w of the object itself are respectively:

[0083] boxself_w = (box_bg_w * (bg_dis * boxself_h)) / bg_dis

[0084] boxself_l = (box_bg_l * (bg_dis * boxself_h)) / bg_dis

[0085] Wherein, bg_dis is the distance H between the ultrasonic module and the conveyor belt measured in real time, and boxself_h is the distance h between the ultrasonic measurement and the upper surface of the object.

[0086] A method for measuring the length, width and height of an object in an industrial assembly line proposed by an embodiment of the present invention performs ultrasonic ranging on an object on the conveyor belt of the assembly line to obtain the height information of the object; performs image acquisition on the object, performs object image segmentation on the acquired object image information by using the GrabCut algorithm, and obtains the length and width of the object in the image; uses the principle of similar triangles to model the object to obtain the length and width information of the object. It can realize the intelligent detection of the size of warehouse packages, with high detection efficiency and good accuracy.

[0087] Embodiment 2

[0088] Corresponding to the above Embodiment 1, as Figure 5 shown, this embodiment proposes a device for measuring the length, width and height of an object in an industrial assembly line, and the device includes:

[0089] An ultrasonic ranging module 210 is used to perform ultrasonic ranging on an object on a conveyor belt of an assembly line to obtain the height information of the object;

[0090] An image acquisition module 220 is used to acquire an image of the object;

[0091] A processing module 230 is used to perform image segmentation on the acquired object image information by using the GrabCut algorithm and obtain the length and width of the object in the image;

[0092] Using the principle of similar triangles, the object is modeled to obtain the length and width information of the object.

[0093] The functions performed by each component in a device for measuring the length, width, and height of an object used in an industrial assembly line provided in an embodiment of the present invention have been described in detail in the above Embodiment 1, so no further elaboration will be made here.

[0094] Embodiment 3

[0095] Corresponding to the above Embodiment 1, this embodiment proposes a system for measuring the length, width, and height of an object used in an industrial assembly line. The system includes: a processor and a memory;

[0096] The memory is used to store one or more program instructions;

[0097] The processor is used to run one or more program instructions to execute the method described in Embodiment 1 above.

[0098] Embodiment 4

[0099] Corresponding to the above embodiment, this embodiment proposes a computer storage medium. The computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a system for measuring the length, width, and height of an object used in an industrial assembly line to perform the method of Embodiment 1.

[0100] Although the present invention has been described in detail with general descriptions and specific embodiments above, based on the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.

Claims

1. A method for measuring the length, width and height of an object in an industrial assembly line, characterized in that, the method includes: Performing ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object; Performing image acquisition on the object, segmenting the object image using the GrabCut algorithm according to the obtained object image information, and obtaining the length and width of the object in the image; Using the principle of similar triangles to model the object to obtain the length and width information of the object; Using the principle of similar triangles to model the object to obtain the length and width information of the object, specifically including: Obtaining the pixel values camera_pix_l and camera_pix_w of the camera, and obtaining the focal point coordinates (f_x, f_y) and the center point coordinates (x0, y0) of the camera according to the internal parameter matrix of the camera; The maximum length bg_l and width bg_w projected by the camera on the conveyor belt are respectively: bg_w = (bg_dis * x0) / f_x bg_l = (bg_dis * y0) / f_y; Then the length each_pix_l and width each_pix_w of each pixel value corresponding to the maximum length and width projected by the camera on the conveyor belt are respectively: each_pix_w = bg_w / camera_pix_w each_pix_l = bg_l / camera_pix_l; Then the length box_bg_l and width box_bg_w projected by the object on the conveyor belt are respectively: box_bg_w = box_pix_w * each_pix_w box_bg_l = box_pix_l * each_pix_l; Then the length boxself_l and width boxself_w of the object itself are respectively: boxself_w = (box_bg_w * (bg_dis * boxself_h)) / bg_dis boxself_l = (box_bg_l * (bg_dis * boxself_h)) / bg_dis Wherein, bg_dis is the distance H measured in real time by the ultrasonic module between the conveyor belt, and boxself_h is the distance h measured by the ultrasonic wave between the upper surface of the object.

2. The method for measuring the length, width and height of an object in an industrial assembly line according to claim 1, characterized in that, Performing ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object, specifically including: Deploying an ultrasonic module above the conveyor belt to measure the distance H between the conveyor belt in real time; When the object is conveyed under the ultrasonic module, measuring the distance h between the upper surface of the object by ultrasonic wave; The height of the object is H - h.

3. The method for measuring the length, width and height of an object in an industrial assembly line according to claim 1, characterized in that, Performing image acquisition on the object, segmenting the object image using the GrabCut algorithm according to the obtained object image information, and obtaining the length and width of the object in the image, specifically including: The object is imaged by an image acquisition module disposed above the conveyor belt, and the image acquisition module and the ultrasonic module are arranged on the same horizontal line; A rectangular area containing the object is set as the ROI area for the acquired object image, and the area outside the rectangular area is automatically set as the background; A mask image of the same size as the object image is obtained, filled with pixel 0, and foreground and background models are created; Based on the GrabCut algorithm function, image segmentation is performed to obtain the object segmentation image, and the four vertex coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) of the object in the image coordinate system are obtained. The length box_pix_l and width box_pix_w of the image pixel values of the object are respectively:

4. A device for measuring the length, width and height of an object in an industrial assembly line, characterized in that, the device includes: an ultrasonic ranging module for performing ultrasonic ranging on the object on the conveyor belt of the assembly line to obtain the height information of the object; an image acquisition module for acquiring an image of the object; a processing module for performing image segmentation on the acquired object image information by using the GrabCut algorithm and obtaining the length and width of the object in the image; using the principle of similar triangles to model the object to obtain the length and width information of the object; using the principle of similar triangles to model the object to obtain the length and width information of the object, specifically including: obtaining the pixel values camera_pix_l and camera_pix_w of the camera, and obtaining the focal point coordinates (f_x, f_y) and the center point coordinates (x0, y0) of the camera according to the internal parameter matrix of the camera; The maximum length bg_l and width bg_w projected by the camera on the conveyor belt are respectively: bg_w = (bg_dis * x0) / f_x bg_l = (bg_dis * y0) / f_y; Then the length each_pix_l and width each_pix_w of each pixel value corresponding to the maximum length and width projected by the camera on the conveyor belt are respectively: each_pix_w = bg_w / camera_pix_w each_pix_l = bg_l / camera_pix_l; Then the length box_bg_l and width box_bg_w projected by the object on the conveyor belt are respectively: box_bg_w = box_pix_w * each_pix_w box_bg_l = box_pix_l * each_pix_l; Then the length boxself_l and width boxself_w of the object itself are respectively: boxself_w = (box_bg_w * (bg_dis * boxself_h)) / bg_dis boxself_l = (box_bg_l * (bg_dis * boxself_h)) / bg_dis where bg_dis is the distance H measured in real time by the ultrasonic module from the conveyor belt, and boxself_h is the distance h measured by the ultrasonic from the upper surface of the object.

5. A system for measuring the length, width and height of an object in an industrial assembly line, characterized in that, the system includes: a processor and a memory; the memory is used for storing one or more program instructions; the processor is used for running one or more program instructions to execute the method according to any one of claims 1-3.

6. A computer storage medium, characterized in that, the computer storage medium contains one or more program instructions, and the one or more program instructions are used to be executed by a system for measuring the length, width and height of an object in an industrial assembly line to execute the method according to any one of claims 1-3.

Images