Object position data detection method and device and grabbing control system

By using image acquisition units in the production line system to periodically collect the conveyor belt images and perform coordinate data deduplication processing, the problem of repeated product grabbing and PLC servo cycle occupancy is solved, and high-precision and high-efficiency object position detection and Crawl control.

CN119090948APending Publication Date: 2024-12-06SIEMENS (CHINA) CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411203749.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The different shooting field and shooting interval of the existing production line system in the visual image acquisition unit lead to repeated capture of products in multiple photos, and the motion control program occupies a large number of PLC servo periods, resulting in low position data accuracy and efficiency.

Method used

The image acquisition unit periodically collects images on the conveyor belt, determines the coordinate data of the object in the world coordinate system, and performs repeated judgment and deduplication based on the data of the previous cycle, optimizes image acquisition and manipulator capture control, and reduces PLC The amount of program of the servo cycle.

Benefits of technology

It realizes accurate deduplication of repeated objects in the image, improves the grab accuracy and efficiency of the robot, reduces the number of programs in the PLC servo cycle, and saves memory space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090948A_ABST
    Figure CN119090948A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an object position data detection method and device and a grabbing control system. The method comprises the following steps: periodically acquiring images in a shooting area above a conveyor belt through an image acquisition unit, wherein the period is determined according to a preset accumulated position value of the conveyor belt; determining coordinate data of all objects in the world coordinate system in the images collected in each period; based on a preset cumulative position value of the conveyor belt, a spatial relationship between a coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, and coordinate data of each object in the images collected in the current period and the previous period in the world coordinate system, obtaining a preset position value of the conveyor belt; judging whether each object in the image acquired in the current period is repeated with the object in the image acquired in the previous period; and de-weighting the coordinate data of the object in the image collected in the current period in the world coordinate system according to the judgment result. According to the embodiment of the invention, accurate deduplication is carried out on the position data of the repeated objects in the acquired image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of motion control technology, and in particular to a method and device for detecting position data of an object and a grasping control system. Background Art

[0002] The existing production line system usually collects the position data of the products on the conveyor belt through the visual image acquisition unit, pre-processes the collected position data and sends it to the robot to guide it to grab the product. However, the different shooting fields and shooting intervals of the visual image acquisition unit will cause a product to appear in multiple photos, but the robot cannot grab the same item repeatedly. How to accurately remove duplicates is an urgent problem to be solved. In addition, on the entire production line, there are already a large number of motion control programs occupying the servo cycle of the PLC. How to obtain more accurate product position data while occupying the servo cycle of the PLC as little as possible is another urgent problem to be solved. Summary of the invention

[0003] In view of this, the present disclosure provides a method and device for detecting position data of an object, and a grasping control system, for at least partially solving the above technical problems.

[0004] In a first aspect, the present disclosure provides a method for detecting position data of an object, comprising:

[0005] The image acquisition unit periodically acquires images in the shooting area above the conveyor belt, wherein the period is determined according to a preset cumulative position value of the conveyor belt;

[0006] Determine coordinate data of all objects in the image collected in each cycle in a world coordinate system;

[0007] Based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current cycle in the world coordinate system and the coordinate data of each object in the image collected in the previous cycle in the world coordinate system, it is determined whether each object in the image collected in the current cycle is repeated with the object in the image collected in the previous cycle;

[0008] According to the judgment result, the coordinate data of the object in the image collected in the current cycle in the world coordinate system is deduplicated.

[0009] In a possible implementation, the determining whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system further includes:

[0010] Based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current cycle in the world coordinate system and the coordinate data of each object in the image collected in the previous cycle in the world coordinate system, calculate the center point offset of each object in the image collected in the current cycle and each object in the image collected in the previous cycle;

[0011] If the calculated center point offset is within a preset range, it is determined that the object is repeated with the object in the image collected in the previous cycle; otherwise, it is determined that the object is not repeated with the object in the image collected in the previous cycle.

[0012] In a possible implementation, before judging whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, the method further includes:

[0013] The coordinate data of all objects in the image collected in each period in the world coordinate system are sorted in descending order of the X-axis coordinates to obtain a corresponding data sequence; wherein the X-axis direction in the world coordinate system points to the conveying direction of the conveyor belt.

[0014] In a possible implementation, the determining whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system further includes:

[0015] Traverse the objects in the sorted sequence corresponding to the image collected in the current cycle and perform the following operations on the traversed objects until it is determined that the currently traversed object does not repeat with the object in the image collected in the previous cycle, then stop traversing and determine that the object behind the object in the sorted sequence does not repeat with the object in the image collected in the previous cycle:

[0016] Based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of the currently traversed object in the world coordinate system and the coordinate data of each object in the sorting sequence corresponding to the image collected in the previous cycle in the world coordinate system, calculate the center point offset of each object in the sorting sequence corresponding to the image collected in the previous cycle;

[0017] If the calculated center point offset of any object in the sorting sequence corresponding to the object and the image collected in the previous cycle is within a preset range, it is determined that the object is repeated with the object in the image collected in the previous cycle; otherwise, it is determined that the object is not repeated with the object in the image collected in the previous cycle.

[0018] In a possible implementation, the deduplication processing of the coordinate data of the object in the image collected in the current period in the world coordinate system according to the judgment result further includes:

[0019] According to the judgment result, the coordinate data of the object in the image collected in the current cycle that is not repeated with the object in the image collected in the previous cycle in the world coordinate system is entered into the first storage area based on the X-axis coordinate.

[0020] In a possible implementation, before judging whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, the method further includes:

[0021] Determine the acquisition number corresponding to the image acquired in the current cycle;

[0022] If the acquisition time is one time, the coordinate data of all objects in the image acquired in the current cycle in the world coordinate system are entered into the first storage area based on the X-axis coordinate;

[0023] If the acquisition number is less than the preset number, the coordinate data of all objects in the image acquired in the current cycle in the world coordinate system are entered into the second storage area based on the X-axis coordinate;

[0024] If the acquisition number is greater than the preset number, clear the preset group data arranged in front in the second storage area, and enter the coordinate data of all objects in the image acquired in the current cycle in the world coordinate system into the second storage area based on the X-axis coordinate;

[0025] The coordinate data of each object in the image acquired in the previous cycle in the world coordinate system is acquired from the second storage area.

[0026] In a possible implementation, the image acquisition unit periodically acquires images in the shooting area above the conveyor belt, wherein the period is determined according to a preset cumulative position value of the conveyor belt, and further includes:

[0027] Record the image acquisition time; and,

[0028] The step of performing deduplication processing on the coordinate data of the object in the image collected in the current cycle in the world coordinate system according to the judgment result further includes:

[0029] According to the judgment result, the coordinate data of the object in the image collected in the current cycle that does not overlap with the object in the image collected in the previous cycle in the world coordinate system and the corresponding image collection time are entered into the first storage area based on the X-axis coordinate.

[0030] In a possible implementation, the method further includes:

[0031] In response to the grasping control signal of the manipulator, the coordinate data of the object ranked first in the first storage area in the world coordinate system and the corresponding image acquisition time are dequeued, and the corresponding data indicating the real-time grasping position is determined based on the dequeued data and output.

[0032] In a possible implementation manner, determining the coordinate data of all objects in the image collected in each period in the world coordinate system further includes:

[0033] Obtaining coordinate data of all objects in the image acquired in each cycle in the image acquisition unit coordinate system;

[0034] The coordinate data of all objects in the image collected in each cycle in the image collection unit coordinate system are converted into the coordinate data of all objects in the image collected in each cycle in the world coordinate system.

[0035] In a second aspect, the present disclosure provides a device for detecting position data of an object, comprising:

[0036] An image acquisition module, used for periodically acquiring images in a shooting area above the conveyor belt through an image acquisition unit, wherein the period is determined according to a preset cumulative position value of the conveyor belt;

[0037] A first determination module, used to determine the coordinate data of all objects in the image collected in each cycle in the world coordinate system;

[0038] a judgment module, configured to judge whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system;

[0039] The deduplication module is used to perform deduplication processing on the coordinate data of the object in the image collected in the current cycle in the world coordinate system according to the judgment result.

[0040] In a third aspect, the present disclosure provides a grasping control system, the system comprising a first organizational block and a second organizational block, wherein a program scanning priority in the second organizational block is greater than a program scanning priority in the first organizational block;

[0041] The first organizational block includes a program corresponding to the device for detecting the position data of the object described in the second aspect, and the device also includes a second determining module; the deduplication module is further used to enter the coordinate data of the object in the image collected in the current cycle that is not repeated with the object in the image collected in the previous cycle in the world coordinate system into the first storage area based on the X-axis coordinate according to the judgment result; the second determining module is used to respond to the grasping control signal of the manipulator, and determine the corresponding data indicating the real-time grasping position based on the coordinate data of the first-ranked object in the world coordinate system in the first storage area and the corresponding image acquisition time;

[0042] The second organizational block includes a program corresponding to a manipulator control device, and the manipulator control device includes a detection module and an acquisition and output module; wherein the detection module is used to detect whether an object enters a grasping area through a second image acquisition unit, and the acquisition and output module is used to send a grasping control signal of the manipulator to the second determination module based on a detection result of whether the object enters the grasping area detected by the detection module, and obtain data indicating a real-time grasping position corresponding to the object from the second determination module and output it to the manipulator to control the manipulator to perform a grasping operation on the object.

[0043] In a fourth aspect, the present disclosure provides an electronic device, the electronic device comprising: a processor, a communication interface, a memory and a bus, wherein the processor, the communication interface and the memory communicate with each other through the bus;

[0044] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to any one of the methods described in the first aspect.

[0045] In a fifth aspect, the present disclosure provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the processor executes any one of the methods described in the first aspect.

[0046] In the embodiment of the present disclosure, according to the cumulative position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current cycle in the world coordinate system and the coordinate data of each object in the image collected in the previous cycle in the world coordinate system, it is judged whether each object in the image collected in the current cycle is repeated with the object in the image collected in the previous cycle, and the data is deduplicated and stored based on the judgment result, which is not limited by the number of objects captured by a single image, the length of the shooting cycle, the size of the shooting area and other parameters, and can accurately deduplicate the data of repeated objects in the collected image, and the deduplication rate reaches 100%. Furthermore, the first cache area and the second cache area are allocated to the data before deduplication and the data after deduplication, respectively, and the useless data in the cache area is cleared in time, effectively saving memory. In addition, for the application scenario of PLC-based robot grasping control, the program of the robot's motion control device is placed in the servo cycle organization block, and the program of the object's position data detection device is placed in other organization blocks with a scanning priority lower than the servo cycle organization block. This ensures the accuracy and timeliness of the detected object's position data while reducing the amount of servo cycle programs and improving the accuracy of the robot's motion control. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a method for detecting position data of an object according to an embodiment of the present disclosure.

[0048] Figure 2 4 is a structural diagram of a device for detecting position data of an object according to an embodiment of the present disclosure.

[0049] Figure 3 is a structural diagram of a grasping control system according to an embodiment of the present disclosure.

[0050] Figure 4is a structural diagram of an electronic device according to an embodiment of the present disclosure.

[0051] List of reference numerals:

[0052] 200. Object position data detection device; 202. Image acquisition module;

[0053] 204, first determination module; 206, judgment module;

[0054] 208, deduplication module; 210, second determination module;

[0055] 300. Grasping control system; 310. First organizational block;

[0056] 320. Second tissue block; 322. Manipulator control device;

[0057] 3222. Detection module; 3224. Acquisition and output module;

[0058] 400. Electronic device; 402. Processor;

[0059] 404, communication interface; 406, memory;

[0060] 408, bus; 410, program. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions, and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other technical solutions obtained by ordinary technicians in the field belong to the scope of protection of the present disclosure.

[0062] Figure 1 The object position data detection method of the embodiment of the present disclosure is shown, which is mainly used for the position data detection of the object on the conveyor belt. Figure 1 As shown, including:

[0063] In step S102, the image acquisition unit periodically acquires images in the shooting area above the conveyor belt. The period is determined according to the preset cumulative position value of the conveyor belt. Specifically, the conveyor belt is driven by a motor system. For example, the position value of the conveyor belt can be recorded in real time by an encoder on the output shaft side of the motor. The period T is equal to the cumulative time length of the conveyor belt from the first position value to the second position value, and the second position value = the first position value + the preset cumulative position value of the conveyor belt. When the image acquisition unit adopts a camera, based on the period T, the shutter of the camera is controlled by the output cam to set its photo interval, interval length, single photo shooting time, etc.

[0064] In addition, the image acquisition unit is usually installed above the conveyor belt, and the shooting area is related to the installation position of the image acquisition unit, but the method of this embodiment is not limited by the size and position of the shooting area.

[0065] Afterwards, the process proceeds to step S104 to determine the coordinate data of all objects in the image collected in each cycle in the world coordinate system.

[0066] In some embodiments, step S104 can be further implemented as follows: S104a, obtaining the coordinate data of all objects in the image collected in each cycle in the image acquisition unit coordinate system; S104b, converting the coordinate data of all objects in the image collected in each cycle in the image acquisition unit coordinate system into the coordinate data of all objects in the image collected in each cycle in the world coordinate system. In other embodiments, if the image acquisition unit coordinate system coincides with the world coordinate system, the coordinate data of all objects in the image collected in each cycle in the world coordinate system can be directly obtained based on the coordinate data of all objects in the image collected in each cycle in the image acquisition unit coordinate system.

[0067] After that, enter step S106, based on the preset cumulative position value of the conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, determine whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period.

[0068] The “coordinate axis of the conveying direction of the conveyor belt” refers to a one-dimensional coordinate axis constructed based on the conveying direction of the conveyor belt.

[0069] There are various specific implementations of step S106. Two specific implementations are listed below, but it can be understood that they are only exemplary and the present embodiment is not limited to these two specific implementations.

[0070] For example, in a specific implementation, step S106 is further implemented as follows: S106a, based on the preset cumulative position value of the conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, calculate the center point offset of each object in the image collected in the current period and each object in the image collected in the previous period; S106b, if the calculated center point offset is within a preset range, determine that the object is repeated with the object in the image collected in the previous period; otherwise, determine that the object is not repeated with the object in the image collected in the previous period.

[0071] For another example, in another specific implementation, before step S106, the following steps are also included: sorting the coordinate data of all objects in the image collected in each period in the world coordinate system in the order of the X-axis coordinate from large to small, to obtain a corresponding data sequence; wherein the X-axis direction in the world coordinate system points to the conveying direction of the conveyor belt. Furthermore, in this implementation, step S106 can be further implemented as: traversing the objects in the sorting sequence corresponding to the image collected in the current cycle and performing the operation "i. Based on the preset cumulative position value of the conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of the currently traversed object in the world coordinate system and the coordinate data of each object in the sorting sequence corresponding to the image collected in the previous cycle in the world coordinate system, calculate the center point offset of each object in the sorting sequence corresponding to the image collected in the previous cycle; ii. If the calculated center point offset of any object in the sorting sequence corresponding to the image collected in the previous cycle is within a preset range, it is determined that the object is repeated with the object in the image collected in the previous cycle; otherwise, it is determined that the object is not repeated with the object in the image collected in the previous cycle", until it is determined that the currently traversed object is not repeated with the object in the image collected in the previous cycle, then the traversal is stopped and it is determined that the object behind the object in the sorting sequence is not repeated with the object in the image collected in the previous cycle. The preset range can be set based on experience.

[0072] It can be understood that although the two specific implementation methods of step S106 listed above both determine whether there is duplication by calculating and comparing the center point offset between objects, it can be understood that the present application is not limited to the center point offset, but can also be a simple variation thereof, for example, it can also be achieved by calculating and comparing the center point distance between objects, etc., which will not be elaborated here.

[0073] It can be understood that in practical applications, for the sake of simplicity in calculation, it is preferred that when constructing the world coordinate system XYZ, its X axis is parallel to the "coordinate axis with the conveying direction of the conveyor belt as the positive direction", and the XY coordinate plane of the world coordinate system is parallel to the conveying surface of the conveyor belt. The following is an example of the preferred exemplary spatial relationship between the "coordinate axis with the conveying direction of the conveyor belt as the positive direction" and the world coordinate system. The formula can be used: Calculate the center point offset of any two objects, where (x1, y1, z1) is the coordinate of the first object in the world coordinate system, (x2, y2, z2) is the coordinate of the second object in the world coordinate system, P is the cumulative position value of the preset conveyor belt, and D is the center point offset of the first object and the second object.

[0074] Afterwards, the process proceeds to step S108 , and according to the judgment result, the coordinate data of the objects in the image collected in the current cycle in the world coordinate system are deduplicated.

[0075] In some embodiments, step S108 can be further implemented as follows: based on the judgment result, the coordinate data of the objects in the image collected in the current period that do not overlap with the objects in the image collected in the previous period in the world coordinate system are queued into the first storage area based on the X-axis coordinate to achieve a deduplication effect.

[0076] In some embodiments, the image acquisition time is also recorded in step S102. And in step S108, the coordinate data of the object in the image collected in the current cycle that is not repeated with the object in the image collected in the previous cycle and the corresponding recorded image acquisition time are queued in the first storage area based on the X-axis coordinate; further, the method of this embodiment also includes: in response to the grasping control signal of the manipulator, the coordinate data of the object currently ranked first in the first storage area in the world coordinate system and the corresponding image acquisition time are dequeued, and the corresponding data indicating the real-time grasping position is determined based on the dequeued data and output (for example, output to the manipulator or to the control device of the manipulator).

[0077] In this embodiment, the first storage area and the second storage area are usually in the form of cache areas. Therefore, in order to save memory space, the data in the first storage area is immediately cleared once it is acquired by other modules or devices (such as the "dequeue" step mentioned above, that is, the data in the first storage area is cleared in time); the data in the second storage area is cleared on demand, and two examples are listed below.

[0078] For example, in some embodiments, before step S106, the following steps are also included: determining the acquisition number corresponding to the image acquired in the current period; if the acquisition number is one, entering the coordinate data of all objects in the image acquired in the current period in the world coordinate system into the first storage area based on the X-axis coordinate; if the acquisition number is less than the preset number, entering the coordinate data of all objects in the image acquired in the current period in the world coordinate system into the second storage area based on the X-axis coordinate; if the acquisition number is greater than the preset number, clearing the preset group data arranged in front in the second storage area, and entering the coordinate data of all objects in the image acquired in the current period in the world coordinate system into the second storage area based on the X-axis coordinate; obtaining the coordinate data of each object in the image acquired in the previous period in the world coordinate system from the second storage area; wherein the preset number and the preset group data can be determined according to the storage capacity of the second storage area and the maximum number of objects in a single image captured. For another example, in some other exemplary embodiments, before step S106, the following steps may also be included: based on the judgment result in step S106, marking the objects in the images collected in the current cycle that are determined to be repeated with the objects in the images collected in the previous cycle, wherein different information is marked for different objects; determining the number of times the objects in the images collected in the current cycle are currently photographed based on the marked information; if the number of times the objects in the images collected in the current cycle are currently photographed is greater than a preset number, deleting the coordinate data of the object marked in the cache queue in the world coordinate system; wherein the preset number is preferably the maximum number of times the same object is repeatedly photographed.

[0079] In order to implement the object position data detection method of the above embodiment, the embodiment of the present disclosure also provides an object position data detection device 200, such as Figure 2 As shown, the device 200 includes an image acquisition module 202, a first determination module 204, a judgment module 206 and a deduplication module 208. It should be noted that since the following embodiments are intended to implement the aforementioned method embodiments, each module in the device 200 is provided to implement each step of the aforementioned method. Therefore, the present invention is not limited to the following embodiments, and any device or module that can implement the aforementioned method should be included in the protection scope of the present invention. In addition, since the device 200 of this embodiment is a device corresponding to the embodiment of the aforementioned detection method, all technical details in the embodiment of the aforementioned detection method are applicable to the device 200 of this embodiment. In order to save space, some of the same contents as the aforementioned method are omitted here.

[0080] like Figure 2As shown, the image acquisition module 202 is used to periodically acquire images in the shooting area above the conveyor belt through the image acquisition unit; wherein the period is determined according to the preset cumulative position value of the conveyor belt. The first determination module 204 is used to determine the coordinate data of all objects in the image acquired in each period in the world coordinate system. The judgment module 206 is used to determine whether each object in the image acquired in the current period is repeated with the object in the image acquired in the previous period based on the preset cumulative position value of the conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the preset cumulative position value of the conveyor belt, the coordinate data of each object in the image acquired in the current period in the world coordinate system, and the coordinate data of each object in the image acquired in the previous period in the world coordinate system. The deduplication module 208 is used to perform deduplication processing on the coordinate data of the objects in the image acquired in the current period in the world coordinate system according to the judgment result.

[0081] In addition, the embodiment of the present disclosure also provides a grasping control system 300, such as Figure 3 As shown, the system 300 includes a first organizational block 310 and a second organizational block 320 , and the program scanning priority in the second organizational block 320 is greater than the program scanning priority in the first organizational block 310 .

[0082] The first organization block 310 includes a program corresponding to the device 200 of the aforementioned embodiment, and the device 200 also includes a second determination module 210, the second determination module 210 is used to determine the corresponding data indicating the real-time grasping position based on the coordinate data of the first-arranged object in the world coordinate system in the first storage area and the recorded image acquisition time in response to the grasping control signal of the manipulator. The second organization block 320 includes a program corresponding to the manipulator control device 322, and the device 322 includes a detection module 3222 and an acquisition and output module 3224; wherein the detection module 3222 is used to detect whether the object enters the grasping area through the second image acquisition unit, and the acquisition and output module 3224 is used to send the grasping control signal of the manipulator to the second determination module based on the detection result of whether the object enters the grasping area detected by the detection module 3222, and obtain the data indicating the real-time grasping position corresponding to the object from the second determination module and output it to the manipulator to control the manipulator to perform a grasping operation on the object.

[0083] Taking PLC as an example, the first organization block 310 is the PLC servo cycle organization block, the second organization block 320 is the OB1 organization block, the program of the device 200 is called in OB1, and the programs of the detection module 3222 and the acquisition and output module 3224 on the robot side are called in the servo cycle. This ensures the accuracy and timeliness of the position data of the detected object while reducing the program amount of the servo cycle and improving the accuracy of the robot's motion control.

[0084] It can be understood that, since the first organizational block 310 includes the program corresponding to the apparatus 200 of the aforementioned embodiment, all technical details in the apparatus 200 of the aforementioned embodiment are applicable to the system 300 of this embodiment and are omitted here for the sake of brevity.

[0085] Figure 4 FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. It should be noted that the present embodiment does not limit the specific implementation of the electronic device 400. Figure 4 As shown, the electronic device 400 includes: a processor 402, a communication interface 404, a memory 406, and a bus 408. Among them:

[0086] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a bus 408 .

[0087] The communication interface 404 is used to communicate with other electronic devices or servers.

[0088] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in the aforementioned method embodiments.

[0089] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0090] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0091] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory 406, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0092] The program 410 may be specifically used to enable the processor 402 to execute the method in any of the aforementioned embodiments.

[0093] The specific implementation of each step in program 410 can refer to the corresponding description of the corresponding steps and units in the method of the aforementioned embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the aforementioned method embodiment, which will not be repeated here.

[0094] The present disclosure also provides a computer-readable storage medium storing instructions for causing a machine to execute the method embodiments described herein. Specifically, a system or device equipped with a storage medium may be provided, on which a software program code for implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device is enabled to read and execute the program code stored in the storage medium.

[0095] In this case, the program code itself read from the storage medium can realize the function of any one of the above embodiments, and thus the program code and the storage medium storing the program code constitute part of the present disclosure.

[0096] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0097] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0098] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0099] It should be noted that not all steps and modules in the above-mentioned processes and system structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0100] In the above embodiments, the hardware module can be implemented mechanically or electrically. For example, a hardware module can include permanent dedicated circuits or logic (such as special processors, FPGA or ASIC) to complete the corresponding operation. The hardware module can also include programmable logic or circuits (such as general-purpose processors or other programmable processors), which can be temporarily set by software to complete the corresponding operation. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily set circuit) can be determined based on cost and time considerations.

[0101] It should be noted that in the claims and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0102] While the present disclosure has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure.

[0103] The present disclosure has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present disclosure is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present disclosure, and these embodiments are also within the protection scope of the present disclosure.

Claims

1. A method for detecting position data of an object, characterized in that: include: The image acquisition unit periodically acquires images in the shooting area above the conveyor belt, wherein the period is determined according to a preset cumulative position value of the conveyor belt; Determine coordinate data of all objects in the image collected in each cycle in a world coordinate system; Based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current cycle in the world coordinate system and the coordinate data of each object in the image collected in the previous cycle in the world coordinate system, it is determined whether each object in the image collected in the current cycle is repeated with the object in the image collected in the previous cycle; According to the judgment result, the coordinate data of the object in the image collected in the current cycle in the world coordinate system is deduplicated.

2. The method according to claim 1, characterized in that The step of judging whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, further comprises: Based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current cycle in the world coordinate system and the coordinate data of each object in the image collected in the previous cycle in the world coordinate system, calculate the center point offset of each object in the image collected in the current cycle and each object in the image collected in the previous cycle; If the calculated center point offset is within a preset range, it is determined that the object is repeated with the object in the image collected in the previous cycle; otherwise, it is determined that the object is not repeated with the object in the image collected in the previous cycle.

3. The method according to claim 1, characterized in that Before judging whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, the method further includes: The coordinate data of all objects in the image collected in each period in the world coordinate system are sorted in descending order of the X-axis coordinates to obtain a corresponding data sequence; wherein the X-axis direction in the world coordinate system points to the conveying direction of the conveyor belt.

4. The method according to claim 3, characterized in that: The step of judging whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, further comprises: Traverse the objects in the sorted sequence corresponding to the image collected in the current cycle and perform the following operations on the traversed objects until it is determined that the currently traversed object does not repeat with the object in the image collected in the previous cycle, then stop traversing and determine that the object behind the object in the sorted sequence does not repeat with the object in the image collected in the previous cycle: Based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of the currently traversed object in the world coordinate system and the coordinate data of each object in the sorting sequence corresponding to the image collected in the previous cycle in the world coordinate system, calculate the center point offset of each object in the sorting sequence corresponding to the image collected in the previous cycle; If the calculated center point offset of any object in the sorting sequence corresponding to the object and the image collected in the previous cycle is within a preset range, it is determined that the object is repeated with the object in the image collected in the previous cycle; otherwise, it is determined that the object is not repeated with the object in the image collected in the previous cycle.

5. The method according to any one of claims 1 to 4, characterized in that The step of performing deduplication processing on the coordinate data of the object in the image collected in the current cycle in the world coordinate system according to the judgment result further includes: According to the judgment result, the coordinate data of the object in the image collected in the current cycle that is not repeated with the object in the image collected in the previous cycle in the world coordinate system is entered into the first storage area based on the X-axis coordinate.

6. The method according to claim 5, characterized in that Before judging whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system, the method further includes: Determine the acquisition number corresponding to the image acquired in the current cycle; If the acquisition time is one time, the coordinate data of all objects in the image acquired in the current cycle in the world coordinate system are entered into the first storage area based on the X-axis coordinate; If the acquisition number is less than the preset number, the coordinate data of all objects in the image acquired in the current cycle in the world coordinate system are entered into the second storage area based on the X-axis coordinate; If the acquisition number is greater than the preset number, clear the preset group data arranged in front in the second storage area, and enter the coordinate data of all objects in the image acquired in the current cycle in the world coordinate system into the second storage area based on the X-axis coordinate; The coordinate data of each object in the image acquired in the previous cycle in the world coordinate system is acquired from the second storage area.

7. The method according to claim 5, characterized in that The image acquisition unit periodically acquires images in the shooting area above the conveyor belt, wherein the period is determined according to a preset cumulative position value of the conveyor belt, and further comprises: Record the image acquisition time; and, The step of performing deduplication processing on the coordinate data of the object in the image collected in the current cycle in the world coordinate system according to the judgment result further includes: According to the judgment result, the coordinate data of the object in the image collected in the current cycle that does not overlap with the object in the image collected in the previous cycle in the world coordinate system and the corresponding image collection time are entered into the first storage area based on the X-axis coordinate.

8. The method according to claim 7, characterized in that The method further comprises: In response to the grasping control signal of the manipulator, the coordinate data of the object ranked first in the first storage area in the world coordinate system and the corresponding image acquisition time are dequeued, and the corresponding data indicating the real-time grasping position is determined based on the dequeued data and output.

9. The method according to claim 1, characterized in that: The determining of the coordinate data of all objects in the image collected in each cycle in the world coordinate system further comprises: Obtaining coordinate data of all objects in the image acquired in each cycle in the image acquisition unit coordinate system; The coordinate data of all objects in the image collected in each cycle in the image collection unit coordinate system are converted into the coordinate data of all objects in the image collected in each cycle in the world coordinate system.

10. A device for detecting position data of an object, characterized in that: The device (200) comprises: An image acquisition module (202) is used to periodically acquire images in a shooting area above the conveyor belt through an image acquisition unit, wherein the period is determined according to a preset cumulative position value of the conveyor belt; A first determination module (204) is used to determine the coordinate data of all objects in the image collected in each cycle in the world coordinate system; A judgment module (206) is used to judge whether each object in the image collected in the current period is repeated with the object in the image collected in the previous period based on the accumulated position value of the preset conveyor belt, the spatial relationship between the coordinate axis of the conveying direction of the conveyor belt and the world coordinate system, the coordinate data of each object in the image collected in the current period in the world coordinate system, and the coordinate data of each object in the image collected in the previous period in the world coordinate system; The deduplication module (208) is used to perform deduplication processing on the coordinate data of the object in the image collected in the current cycle in the world coordinate system according to the judgment result.

11. A grasping control system, characterized in that: The system (300) comprises a first organization block (310) and a second organization block (320), wherein a program scanning priority in the second organization block (320) is greater than a program scanning priority in the first organization block (310); The first organization block (310) includes a program corresponding to the object position data detection device (200) according to claim 10, and the device (200) further includes a second determination module (210); the deduplication module (208) is further used to enter the coordinate data of the object in the image collected in the current cycle that is not repeated with the object in the image collected in the previous cycle in the world coordinate system into the first storage area based on the X-axis coordinate according to the judgment result; the second determination module (210) is used to respond to the grasping control signal of the manipulator, and determine the corresponding data indicating the real-time grasping position based on the coordinate data of the first-ranked object in the world coordinate system in the first storage area and the corresponding image acquisition time; The second organizational block (320) includes a program corresponding to a manipulator control device (322), and the manipulator control device (322) includes a detection module (3222) and an acquisition and output module (3224); wherein the detection module (3222) is used to detect whether an object enters a grasping area through a second image acquisition unit, and the acquisition and output module (3224) is used to send a grasping control signal of the manipulator to the second determination module (210) based on a detection result of whether the object enters the grasping area detected by the detection module, and to obtain data indicating a real-time grasping position corresponding to the object from the second determination module (210) and output it to the manipulator to control the manipulator to perform a grasping operation on the object.

12. An electronic device, the electronic device (400) comprising: A processor (402), a communication interface (404), a memory (406) and a bus (408), wherein the processor (402), the communication interface (404) and the memory (406) communicate with each other via the bus (408); The memory (406) is used to store at least one executable instruction, and the executable instruction enables the processor (402) to perform an operation corresponding to the method as described in any one of claims 1 to 9.

13. A computer-readable storage medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Industrial robot grabbing method based on visual following

    CN110948491A

  • Robot vision system

    CN111775154A

  • Linear conveyor belt tracking control method, system and device and storage medium

    CN113269085A

  • Control method of mechanical arm

    CN117944016A

  • Cross reality system with quality information about persistent coordinate frames

    US20210134064A1