Method and apparatus for generating mechanical arm pose positioning information

Abstract

The present disclosure provides a method and device for generating mechanical arm pose positioning information, which can be applied in the field of artificial intelligence and financial technology. The method comprises: in response to a received target image, extracting image features of the target image, wherein the target image is collected by an image collection device on a mechanical arm in the case that a part of the target object is blocked by an obstacle; identifying the image features to obtain first contour information of the target object; wherein the first contour information represents pixel point information of the target object in the area not blocked by the obstacle; generating second contour information of the target object according to the first contour information and size information of the target object, wherein the second contour information represents pixel point information of the edge of the target object; and generating pose positioning information for representing the pose of the mechanical arm when grabbing the target object according to the second contour information based on a coordinate conversion relationship.

Description

Technical Field

[0001] This disclosure relates to the fields of image processing and financial technology, and in particular to a method, apparatus, device, medium and program product for generating robotic arm posture positioning information. Background Technology

[0002] With the continuous development of technology, the automation industry is developing rapidly, and the application range of robotic arms is becoming increasingly wide. In related technologies, the common approach is to use image acquisition devices fixed to the robotic arm to capture images of the object being grasped, and then use image processing to achieve posture positioning of the robotic arm.

[0003] However, in the process of realizing the inventive concept of this disclosure, when the image acquisition device has a blind spot, the acquired object image is incomplete, resulting in low posture positioning accuracy of the robotic arm. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method, apparatus, device, medium and program product for generating robotic arm posture and positioning information.

[0005] According to a first aspect of this disclosure, a method for generating robotic arm posture positioning information is provided, comprising:

[0006] In response to the received target image, the image features of the target image are extracted, wherein the target image is acquired using an image acquisition device on a robotic arm when a portion of the target object is occluded by an obstacle;

[0007] Image features are identified to obtain the first contour information of the target object; the first contour information represents the pixel information of the area of ​​the target object that is not occluded by obstacles;

[0008] Based on the first contour information and the size information of the target object, second contour information of the target object is generated, wherein the second contour information represents the pixel information of the edge of the target object; and

[0009] Based on the coordinate transformation relationship, the posture positioning information used to characterize the robotic arm when grasping the target object is generated according to the second contour information. The coordinate transformation relationship characterizes the transformation relationship between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the robotic arm.

[0010] According to embodiments of this disclosure, image features are identified to obtain first contour information of the target object, including:

[0011] Image features are identified to obtain the first pixel information of the obstacle's edge and the second pixel information of the edge of the area of ​​the target object not occluded by the obstacle; and

[0012] First contour information is generated based on the information of the first pixel and the information of the second pixel.

[0013] According to embodiments of this disclosure, the first pixel information includes coordinate information of M first pixels, where M is an integer greater than 1; the second pixel information includes coordinate information of N second pixels, where N is an integer greater than 1; and the generation of first contour information based on the first pixel information and the second pixel information includes:

[0014] Based on the coordinate information of the nth second pixel and the coordinate information of the M first pixels, generate the distance information between the M first pixels and the nth second pixel;

[0015] Based on distance information, the target pixel is determined from M first pixel points;

[0016] If n is determined to be less than N, return to the operation of generating distance information between the M first pixels and the nth second pixel, and increment n; and

[0017] Given that n equals N, first contour information is generated based on the second pixel information and multiple target pixel information, where n is an integer greater than or equal to 1 and less than N.

[0018] According to embodiments of this disclosure, generating second contour information of the target object based on first contour information and the size information of the target object includes:

[0019] Determine the axis of symmetry of the target object based on the first contour information and the size information of the target object;

[0020] Based on the symmetry axis information, the first contour information is flipped to generate the target contour information; and

[0021] Based on the first contour information and the target contour information, the second contour information is generated.

[0022] According to embodiments of this disclosure, determining the axis of symmetry information of the target object based on first contour information and the size information of the target object includes:

[0023] Based on the first contour information and the size information of the target object, determine the coordinate information of at least two pixels located on the axis of symmetry of the target object from the first contour information; and

[0024] Generate the axis of symmetry information of the target object based on the coordinate information of at least two pixels.

[0025] According to embodiments of this disclosure, based on symmetry axis information, the first contour information is flipped to generate target contour information, including:

[0026] Determine the flipping direction based on the relative positions of the target and obstacles; and

[0027] Based on the symmetry axis information, the first contour information is flipped along the flipping direction to generate the target contour information.

[0028] According to embodiments of this disclosure, based on coordinate transformation relationships and second contour information, posture positioning information characterizing the robotic arm's grasping of a target object is generated, including:

[0029] Determine the relative positional relationship of the gripping points based on the gripping parameters of the robotic arm;

[0030] Based on the relative positional relationship of the grasping points, the target pixel information is obtained from the second contour information; and

[0031] Based on the coordinate transformation relationship, the attitude positioning point information is generated according to the target pixel information.

[0032] A second aspect of this disclosure provides a robotic arm posture positioning information generation device, comprising: an extraction module, an identification module, a first generation module, and a second generation module.

[0033] The extraction module is used to extract image features of the target image in response to the received target image, wherein the target image is acquired by the image acquisition device on the robotic arm when part of the target object is occluded by an obstacle.

[0034] The recognition module is used to recognize image features and obtain the first contour information of the target object; wherein, the first contour information represents the pixel information of the area of ​​the target object that is not occluded by obstacles.

[0035] The first generation module is used to generate second contour information of the target object based on the first contour information and the size information of the target object, wherein the second contour information represents the pixel information of the edge of the target object.

[0036] The second generation module is used to generate posture positioning information that characterizes the robotic arm when grasping the target object based on the coordinate transformation relationship and the second contour information. The coordinate transformation relationship characterizes the transformation relationship between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the robotic arm.

[0037] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method described above.

[0038] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0039] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0040] According to the method and apparatus for generating robotic arm posture positioning information provided in this disclosure, partial contour information of a target object is identified based on an image. Based on the partial contour and size of the target object, complete contour information of the target object is generated. Then, based on the transformation relationship between pixel coordinates and spatial coordinates, accurate positioning of the grasping posture is achieved. Therefore, this method at least partially solves the problem of low robotic arm posture positioning accuracy caused by incomplete objects in images, achieving the technical effect of accurately positioning the robotic arm posture. Attached Figure Description

[0041] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0042] Figure 1 This diagram illustrates an application scenario of a method for generating robotic arm posture and positioning information according to an embodiment of the present disclosure.

[0043] Figure 2 A flowchart illustrating a method for generating robotic arm posture and positioning information according to an embodiment of the present disclosure is shown schematically.

[0044] Figure 3 A schematic diagram illustrating the posture positioning process of a robotic arm according to an embodiment of the present disclosure is shown.

[0045] Figure 4 This schematic diagram illustrates a target object occluded by an obstacle according to an embodiment of the present disclosure.

[0046] Figure 5 This schematic diagram illustrates the generation of second contour information of a target object according to an embodiment of the present disclosure;

[0047] Figure 6 A schematic diagram illustrating the fusion of a robotic arm and vision according to an embodiment of the present disclosure is shown.

[0048] Figure 7 A schematic block diagram of a robotic arm posture positioning information generation device according to an embodiment of the present disclosure is shown; and

[0049] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for generating robotic arm posture positioning information according to an embodiment of the present disclosure. Detailed Implementation

[0050] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0051] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0052] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0053] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0054] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0055] In related technologies, images acquired using image acquisition devices fixed on robotic arms have certain visual blind spots, and may only be able to acquire partial images of the target object when the target object is obscured.

[0056] Therefore, when performing robotic arm posture localization based on local images, the shape of the target object cannot be fully displayed in the local image, resulting in a certain error in the robotic arm posture localization.

[0057] In view of this, embodiments of the present disclosure provide a method for generating posture positioning information of a robotic arm, comprising: in response to a received target image, extracting image features of the target image, wherein the target image is acquired using an image acquisition device on the robotic arm when a portion of the target object is occluded by an obstacle; identifying the image features to obtain first contour information of the target object; the first contour information representing pixel information of the unoccluded area of ​​the target object; generating second contour information of the target object based on the first contour information and the size information of the target object, wherein the second contour information represents pixel information of the edges of the target object; and generating posture positioning information representing the robotic arm grasping the target object based on the second contour information according to a coordinate transformation relationship, wherein the coordinate transformation relationship represents the transformation relationship between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the robotic arm.

[0058] Figure 1 The diagram illustrates an application scenario of a method for generating robotic arm posture and positioning information according to an embodiment of the present disclosure.

[0059] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0060] Users can interact with server 105 via network 104 using at least one of the first terminal device 101, second terminal device 102, and third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, second terminal device 102, and third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0061] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0062] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0063] It should be noted that the method for generating robotic arm posture positioning information provided in this embodiment can generally be executed by server 105. Correspondingly, the device for generating robotic arm posture positioning information provided in this embodiment can generally be located in server 105. The method for generating robotic arm posture positioning information provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the device for generating robotic arm posture positioning information provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0064] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0065] The following will be based on Figure 1 The described scene, through Figures 2-6 The method for generating robotic arm posture and positioning information according to the disclosed embodiments is described in detail.

[0066] Figure 2 A flowchart illustrating a method for generating robotic arm posture and positioning information according to an embodiment of the present disclosure is shown.

[0067] like Figure 2 As shown, the method for generating robotic arm posture positioning information in this embodiment includes operations S210 to S240.

[0068] In operation S210, in response to the received target image, image features of the target image are extracted.

[0069] In operation S220, image features are identified to obtain the first contour information of the target object; wherein, the first contour information represents the pixel information of the area of ​​the target object that is not occluded by obstacles.

[0070] In operation S230, based on the first contour information and the size information of the target object, a second contour information of the target object is generated, wherein the second contour information represents the pixel information of the edge of the target object.

[0071] In operation S240, based on the coordinate transformation relationship and according to the second contour information, posture positioning information is generated to characterize the robotic arm when grasping the target object. The coordinate transformation relationship characterizes the transformation relationship between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the robotic arm.

[0072] According to embodiments of this disclosure, the target image is acquired using an image acquisition device on a robotic arm when a portion of the target object is obscured by an obstacle. For example, image features may include information such as pixel color, contrast, brightness, and coordinates.

[0073] According to embodiments of this disclosure, a pre-trained object detection model can be used to identify image features and obtain the first contour information of the target object. The first contour information of the target object, for example, is the contour of the front view of a cube-shaped wooden block, which is the side length of a square. If the target object is occluded, the first contour can be generated based on the edges of the obstacle and the edges of the target object that are not occluded by the obstacle.

[0074] According to an embodiment of this disclosure, the second contour information of the target object is generated by determining the axis of symmetry of the target object from the first contour information and the size information of the target object, and then flipping it up and down (left and right) along the axis of symmetry according to the actual position.

[0075] According to embodiments of this disclosure, attitude positioning information can characterize the relative position information of multiple gripping points of a robotic arm used to grasp a target object.

[0076] According to embodiments of this disclosure, the coordinate transformation relationship can characterize the relationship matrix between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the environment in which the target object is located. This relationship matrix can be obtained by calibration based on a predetermined standard image and a predetermined standard object of the environment in which the target object is located.

[0077] According to embodiments of this disclosure, partial contour information of a target object is identified based on an image. Based on the partial contour and size of the target, complete contour information of the target object is generated. Then, based on the transformation relationship between pixel coordinates and spatial coordinates, accurate positioning of the grasping posture is achieved. Therefore, this at least partially solves the problem of low posture positioning accuracy of the robotic arm due to incomplete objects in the image, achieving the technical effect of accurately positioning the robotic arm posture.

[0078] Figure 3 A schematic diagram illustrating the posture positioning process of a robotic arm according to an embodiment of the present disclosure is shown.

[0079] like Figure 3 As shown, in embodiment 300, a robotic arm 310 and an image acquisition device 320 mounted on the robotic arm 310 may be included. The image acquisition device 320 can acquire image information of the target object, and based on the image information of the target object, the posture positioning information of the robotic arm can be obtained using a nine-point positioning method.

[0080] It should be noted that the nine-point positioning method is a relatively mature method for locating the posture of a robotic arm using images, and will not be elaborated on here.

[0081] According to embodiments of this disclosure, identifying image features to obtain first contour information of a target object may include the following operations: identifying image features to obtain first pixel information of the edge of an obstacle and second pixel information of the edge of the area of ​​the target object not occluded by the obstacle; and generating first contour information based on the first pixel information and the second pixel information.

[0082] Figure 4 The illustration shows a schematic diagram of a target object being obscured by an obstacle according to an embodiment of the present disclosure.

[0083] like Figure 4 As shown, in embodiment 400, a target 410 and an obstacle 420 may be included.

[0084] The outline of the target object 410 can be abcd, but when the target object is occluded by the obstacle 420, the outline of the target object cannot be identified and obtained when the image features are recognized. In this case, the outline information of the part of the target object that is not occluded by the obstacle can be obtained first through image feature recognition.

[0085] According to embodiments of this disclosure, the edge point where the target object is obscured by an obstacle can be determined based on the distance between two coordinate points.

[0086] For example: the first pixel information includes the coordinate information of M first pixels, where M is an integer greater than 1; the second pixel information includes the coordinate information of N second pixels, where N is an integer greater than 1; based on the first pixel information and the second pixel information, first contour information is generated, including:

[0087] Based on the coordinate information of the nth second pixel and the coordinate information of the M first pixels, the distance information between the M first pixels and the nth second pixel is obtained by iterative calculation.

[0088] The distance between any two points A(x1, y1) and B(x2, y2) is calculated as shown in formula (1):

[0089]

[0090] Based on distance information, the target pixel is determined from M first pixel points;

[0091] If n is determined to be less than N, return to the operation of generating distance information between the M first pixels and the nth second pixel, and increment n; and

[0092] Given that n equals N, first contour information is generated based on the second pixel information and multiple target pixel information, where n is an integer greater than or equal to 1 and less than N.

[0093] like Figure 4 As shown, based on the comparison of pixel distances between visible pixels on the target and the obstacle, the edge of the target can be determined as the pixels in line BP. Therefore, the first contour information can be abBP. This includes the first pixel of the edge BP where the obstacle 420 overlaps with the target 410, and the second pixel of the edge abBP of the unoccluded area of ​​the target object. Based on the first and second pixel coordinates, the first contour coordinate information is generated.

[0094] According to embodiments of this disclosure, based on the distance between pixels, the overlapping edge of the target object and the obstacle can be found easily and quickly, thereby determining the first contour information more accurately.

[0095] According to an embodiment of this disclosure, generating second contour information of the target object based on first contour information and the size information of the target object includes: determining the axis of symmetry information of the target object based on the first contour information and the size information of the target object; performing a flipping process on the first contour information based on the axis of symmetry information to generate target contour information; and generating second contour information based on the first contour information and the target contour information.

[0096] According to embodiments of this disclosure, in practical application scenarios, the target object gripped by the robotic arm is usually geometrically symmetrical. Therefore, based on the principle of geometric symmetry, the contour of the occluded part of the target object can be symmetrically restored according to the contour of the unoccluded part of the target object.

[0097] However, since the position of the obstacle obscuring the target object is not necessarily along the axis of symmetry, it is necessary to accurately determine the position of the axis of symmetry.

[0098] For example, determining the axis of symmetry information of a target object based on the first contour information and the size information of the target object includes: determining the coordinate information of at least two pixels located on the axis of symmetry of the target object from the first contour information based on the first contour information and the size information of the target object; and generating the axis of symmetry information of the target object based on the coordinate information of the at least two pixels.

[0099] According to an embodiment of this disclosure, a target contour information is generated by flipping the first contour information based on the axis of symmetry information, including: determining the flipping direction based on the relative position of the target object and the obstacle; and flipping the first contour information along the flipping direction based on the axis of symmetry information to generate the target contour information.

[0100] Figure 5 A schematic diagram illustrating the generation of second contour information of a target object according to an embodiment of the present disclosure is shown.

[0101] like Figure 5 As shown, a first contour 510abPQ is generated based on the edge PQ of the obstacle and the edge of the target object that is not occluded by the obstacle; it can be flipped left and right based on the axis of symmetry L520. The width of the object is ac = 10cm. Based on the position coordinates of point a, while ensuring that the ordinate value remains unchanged, the point L on the axis of symmetry... i The shortest distance from point a is 5cm. The coordinates of point L1 on the axis of symmetry can be calculated. Similarly, the distance from point b to point L2 on the axis of symmetry is obtained. The axis of symmetry L is determined using points L1 and L2: aX + b. Based on the first contour information and the axis of symmetry information, the specific location of the occluded target object and its relative position to the target object are first determined. Along the axis of symmetry L, the second contour information is obtained by flipping the object along the axis of symmetry. For example, if the right side of the target object is occluded, the axis of symmetry L is found based on the first contour abPQ and the size of the target object. Since the right side of the target object is occluded, the first contour abPQ is flipped to the right along the axis of symmetry L to obtain the second contour abcd530.

[0102] According to embodiments of this disclosure, based on the geometric symmetry of the target object, the contour information of the entire target object is obtained by symmetrically restoring the unobstructed area of ​​the target object. This enables accurate positioning of the robotic arm's posture information even when the target object is obstructed, thereby improving the robotic arm's grasping accuracy.

[0103] According to embodiments of this disclosure, generating posture positioning information to characterize the robotic arm grasping a target object based on coordinate transformation relationships and second contour information may include: determining the relative positional relationship of the grasping points based on the grasping parameters of the robotic arm; obtaining target pixel information from the second contour information based on the relative positional relationship of the grasping points; and generating posture positioning point information based on the target pixel information and coordinate transformation relationships.

[0104] According to embodiments of this disclosure, the gripping parameters of the robotic arm may include the number of claws of the robotic arm and the range of relative positions between each claw. For example, the robotic arm may be a four-claw robotic arm, and the four claws may be arranged in a cross shape. The minimum and maximum distances between two claws located on the same horizontal line can represent the gripping range of the robotic arm, that is, the relative positional relationship of the gripping points.

[0105] According to embodiments of this disclosure, the coordinate transformation relationship can be a coordinate transformation matrix obtained based on the camera coordinate system of the image acquisition device and the spatial coordinate system of the calibration object. Based on the coordinate transformation matrix, the target pixels are transformed to generate the posture and positioning information of the robotic arm.

[0106] Figure 6 A schematic diagram illustrating the fusion of a robotic arm and vision according to an embodiment of the present disclosure is shown.

[0107] like Figure 6 As shown, the system includes a vision unit 610 and a robot unit 620. The vision unit 610 may include: a camera 611, a camera system 612, image coordinates of the target 613, a transformation of the target relative to the camera coordinates 614, and a transformation from the camera coordinates to the robot's base coordinates 615. The robot unit may include: robot inverse kinematics solution 621, robot controller control 622, and robot grasping 623.

[0108] Based on the image information acquired by the vision component, the robotic arm posture positioning information is obtained according to the method for generating robotic arm posture positioning information provided in this embodiment. The robotic arm posture positioning information can be sent to the robot component in the form of instructions, allowing the robot to perform the grasping action on the target object according to the robotic arm posture positioning information.

[0109] Based on the above-described method for generating robotic arm posture positioning information, this disclosure also provides a robotic arm posture positioning information generation device. The following will be combined with... Figure 7 The device is described in detail.

[0110] Figure 7 A schematic block diagram of a robotic arm posture positioning information generation device according to an embodiment of the present disclosure is shown.

[0111] like Figure 7 As shown, the robotic arm posture positioning information generation device 700 of this embodiment includes an extraction module 710, an identification module 720, a first generation module 730, and a second generation module 740.

[0112] The extraction module 710 is used to extract image features of the target image in response to the received target image. In one embodiment, the extraction module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0113] The recognition module 720 is used to recognize image features to obtain the first contour information of the target object. In one embodiment, the recognition module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0114] The first generation module 730 is used to generate second contour information of the target object based on the first contour information and the size information of the target object. In one embodiment, the first generation module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0115] The second generation module 740 is used to generate posture and positioning information characterizing the robotic arm when grasping a target object, based on the coordinate transformation relationship and the second contour information. In one embodiment, the second generation module 740 can be used to perform the operation S740 described above, which will not be repeated here.

[0116] According to embodiments of this disclosure, the recognition module 720 may include a recognition submodule and a first generation submodule. The recognition submodule is used to recognize image features to obtain first pixel information of the edge of an obstacle and second pixel information of the edge of the area of ​​the target object not occluded by the obstacle. The first generation submodule is used to generate first contour information based on the first and second pixel information.

[0117] According to embodiments of this disclosure, the generation submodule may include a first generation unit, a first determination unit, a first operation unit, and a second generation unit. The first generation unit is configured to generate distance information between M first pixels and the nth second pixel based on the coordinate information of the nth second pixel and the coordinate information of M first pixels. The first determination unit is configured to determine a target pixel from the M first pixels based on the distance information. The first operation unit is configured to, if n is determined to be less than N, return to the operation of generating the distance information between the M first pixels and the nth second pixel, and increment n. The second generation unit is configured to, if n is determined to be equal to N, generate first contour information based on the second pixel information and multiple target pixel information, where n is an integer greater than or equal to 1 and less than N.

[0118] According to embodiments of this disclosure, the first generation module includes a first determining submodule, a second generation submodule, and a third generation submodule. The first determining submodule is used to determine the axis of symmetry information of the target object based on the first contour information and the size information of the target object. The second generation submodule is used to perform a flipping process on the first contour information based on the axis of symmetry information to generate target contour information. The third generation submodule is used to generate second contour information based on the first contour information and the target contour information.

[0119] According to embodiments of this disclosure, the first determining submodule includes a second determining unit and a third generating unit. The second determining unit is configured to determine the coordinate information of at least two pixels located on the axis of symmetry of the target object from the first contour information based on the first contour information and the size information of the target object. The third generating unit is configured to generate the axis of symmetry information of the target object based on the coordinate information of the at least two pixels.

[0120] According to embodiments of this disclosure, the second generation submodule includes: a third determining unit, configured to determine the flipping direction based on the relative position of the target object and the obstacle; and a fourth generation unit, configured to perform flipping processing on the first contour information along the flipping direction based on the symmetry axis information to generate target contour information.

[0121] According to embodiments of this disclosure, the second generation module includes: a second determining submodule, an obtaining submodule, and a fourth generation submodule. The second determining submodule is used to determine the relative positional relationship of the gripping points based on the gripping parameters of the robotic arm. The obtaining submodule is used to obtain target pixel information from the second contour information based on the relative positional relationship of the gripping points. The fourth generation submodule is used to generate posture positioning point information based on the target pixel information and coordinate transformation relationships.

[0122] According to embodiments of this disclosure, any plurality of modules among the extraction module 710, identification module 720, first generation module 730, and second generation module 740 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the extraction module 710, identification module 720, first generation module 730, and second generation module 740 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the extraction module 710, the recognition module 720, the first generation module 730, and the second generation module 740 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0123] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for generating robotic arm posture positioning information according to an embodiment of the present disclosure.

[0124] like Figure 8As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0125] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0126] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0127] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0128] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0129] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0130] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0131] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0132] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0133] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0136] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for generating robotic arm posture and positioning information, comprising: In response to a received target image, image features of the target image are extracted, wherein the target image is acquired using an image acquisition device on a robotic arm when a portion of the target object is obscured by an obstacle; The image features are identified to obtain the first contour information of the target object; wherein, the first contour information represents the pixel information of the area of ​​the target object that is not occluded by the obstacle; Based on the first contour information and the size information of the target object, second contour information of the target object is generated, wherein the second contour information represents the pixel information of the edge of the target object; and Based on the coordinate transformation relationship, according to the second contour information, posture positioning information is generated to characterize the robotic arm when grasping the target object, wherein the coordinate transformation relationship characterizes the transformation relationship between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the robotic arm; The step of identifying the image features to obtain the first contour information of the target object includes: The image features are identified to obtain the first pixel information of the edge of the obstacle and the second pixel information of the edge of the area of ​​the target object not obscured by the obstacle; and The first contour information is generated based on the first pixel information and the second pixel information; Wherein, the first pixel information includes the coordinate information of M first pixels, where M is an integer greater than 1; the second pixel information includes the coordinate information of N second pixels, where N is an integer greater than 1; the step of generating the first contour information based on the first pixel information and the second pixel information includes: Based on the coordinate information of the nth second pixel and the coordinate information of the M first pixels, the distance information between the M first pixels and the nth second pixel is generated; Based on the distance information, the target pixel is determined from the M first pixel points; If n is determined to be less than N, return to the operation of generating the distance information between the M first pixels and the nth second pixel, and increment n; and Given that n equals N, the first contour information is generated based on the second pixel information and the multiple target pixel information, where n is an integer greater than or equal to 1 and less than N.

2. The method according to claim 1, wherein, The step of generating second contour information of the target object based on the first contour information and the size information of the target object includes: The symmetry axis information of the target object is determined based on the first contour information and the size information of the target object; Based on the symmetry axis information, the first contour information is flipped to generate target contour information; and The second contour information is generated based on the first contour information and the target contour information.

3. The method according to claim 2, wherein, Determining the axis of symmetry information of the target object based on the first contour information and the size information of the target object includes: Based on the first contour information and the size information of the target object, determine the coordinate information of at least two pixels located on the axis of symmetry of the target object from the first contour information; and Based on the coordinate information of the at least two pixels, the symmetry axis information of the target object is generated.

4. The method according to claim 2, wherein, The step of flipping the first contour information based on the symmetry axis information to generate target contour information includes: The flipping direction is determined based on the relative positions of the target and the obstacle; and Based on the symmetry axis information, the first contour information is flipped along the flipping direction to generate the target contour information.

5. The method according to claim 1, wherein, The step of generating posture and positioning information based on coordinate transformation relationships and the second contour information to characterize the robotic arm's grasping of the target object includes: The relative positional relationship of the gripping points is determined based on the gripping parameters of the robotic arm; Based on the relative positional relationship of the grasping points, the target pixel information is obtained from the second contour information; and Based on the coordinate transformation relationship, attitude positioning point information is generated according to the target pixel information.

6. A robotic arm posture positioning information generation device, comprising: An extraction module is used to extract image features of a target image in response to a received target image, wherein the target image is acquired using an image acquisition device on a robotic arm when a portion of the target object is obscured by an obstacle; The recognition module is used to recognize the image features to obtain the first contour information of the target object; wherein, the first contour information represents the pixel information of the area of ​​the target object that is not occluded by the obstacle; A first generation module is configured to generate second contour information of the target object based on the first contour information and the size information of the target object, wherein the second contour information represents the pixel information of the edge of the target object; and The second generation module is used to generate posture positioning information representing the robotic arm grasping the target object based on the second contour information and the coordinate transformation relationship, wherein the coordinate transformation relationship represents the transformation relationship between the pixel coordinate system of the image acquisition device and the spatial coordinate system of the robotic arm. Specifically, the recognition module is used to recognize the image features to obtain first pixel information of the edge of the obstacle and second pixel information of the edge of the area of ​​the target object not obscured by the obstacle; and to generate first contour information based on the first pixel information and the second pixel information; wherein the first pixel information includes coordinate information of M first pixels, where M is an integer greater than 1; the second pixel information includes coordinate information of N second pixels, where N is an integer greater than 1; to generate distance information between the M first pixels and the nth second pixel based on the coordinate information of the nth second pixel and the coordinate information of the M first pixels; to determine the target pixel from the M first pixels based on the distance information; if n is less than N, to return to the operation of generating the distance information between the M first pixels and the nth second pixel, and to increment n; and if n is equal to N, to generate the first contour information based on the second pixel information and multiple target pixel information, where n is an integer greater than or equal to 1 and less than N.

7. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images