Spatial positioning method, biochip moving method and related equipment

Abstract

The invention provides a spatial positioning method, a biochip moving method and related equipment, and the spatial positioning method comprises the steps: obtaining a target image obtained by photographing a marker through a camera, the marker comprising a circle center, a first target point and a second target point; obtaining the number of target pixel points occupied by the diameter of the marker in the target image; determining a first distance between the circle center and the original point of the camera coordinate system according to the number of the target pixel points, the internal reference of the camera and the length parameter; determining a first preset point on a connecting line of the original point and the first target point and a second preset point on a connecting line of the original point and the second target point; determining coordinate representations of the circle center, the first target point, the second target point, the first preset point and the second preset point in a camera coordinate system, and constructing a positioning equation set; and determining coordinates of the circle center, the first target point and the second target point based on the positioning equation set so as to determine the position of the to-be-positioned target object in the camera coordinate system. According to the invention, the precision of spatial positioning can be improved.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular, to a spatial positioning method, a method for moving a biochip based on a robotic arm, and related devices. Background Art

[0002] With the development of industrial technology, the demand for spatial positioning has received increasing attention in many fields such as manufacturing, aerospace, and navigation positioning. In related technologies, Bluetooth positioning technology, ultrasonic positioning technology, infrared positioning technology, laser positioning technology, and WiFi positioning technology are usually used. To achieve spatial positioning using these methods, active emission devices are required, which are difficult to deploy and costly, and are easily interfered with in complex environments. In related technologies, inertial sensors are also used for positioning. However, inertial sensors have the defect of low positioning accuracy. Summary of the Invention

[0003] In view of the above, it is necessary to provide a spatial positioning method, a method for moving a biochip based on a robotic arm, and related devices to solve the technical problem of low accuracy during spatial positioning. Among them, the related devices include a spatial positioning device, an electronic device, and a storage medium.

[0004] This application provides a spatial positioning method, which is applied to an electronic device. The electronic device is communicatively connected to a camera, and the camera is used to capture a target object to be positioned. Among them, an identification object is provided on the target object to be positioned, and the identification object is circular. The method includes: obtaining a target image captured by the camera of the identification object, where the identification object includes a center point, a first target point, and a second target point; obtaining the number of target pixel points occupied by the diameter of the identification object in the target image; determining a first distance between the center point and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the camera, and a pre-stored length parameter; determining a first preset point located on the line connecting the origin and the first target point, where the distance between the first preset point and the origin is equal to the first distance; determining a second preset point located on the line connecting the origin and the second target point, where the distance between the second preset point and the origin is equal to the first distance; determining the coordinate representations of the center point, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; constructing a positioning equation set according to the first distance, the diameter of the identification object, and the coordinate representations; determining the coordinates of the center point, the first target point, and the second target point in the camera coordinate system based on the positioning equation set; and determining the position of the target object to be positioned in the camera coordinate system according to the coordinates.

[0005] In some embodiments, the method for determining the pre-stored pixel point length parameter includes: obtaining a test image captured by the camera for the marker, wherein the distance between the center of the marker and the origin of the camera coordinate system is equal to the test distance; determining the number of test pixel points occupied by the diameter of the marker in the test image; calculating the length represented by each pixel point in the test image in the camera coordinate system according to the internal parameters of the camera, the test distance, and the number of test pixel points, to obtain the pre-stored length parameter.

[0006] In some embodiments, the determining the first distance between the center of the circle and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the camera, and the pre-stored pixel point length parameter includes: calculating the product of the pixel point length parameter and the number of target pixel points to obtain the virtual diameter of the marker in the target image; calculating the ratio between the diameter and the virtual diameter; and obtaining the first distance by calculating the product of the ratio and the internal parameters of the camera.

[0007] In some embodiments, the positioning equation set includes a first equation set, and the method for determining the first equation set includes: determining a first vector according to the coordinate representation of the center of the circle and the coordinate representation of the first target point; determining a second vector according to the coordinate representation of the center of the circle and the coordinate representation of the second target point; determining a third vector according to the coordinate representation of the center of the circle and the coordinate of the origin of the camera coordinate system; determining a fourth vector according to the coordinate representation of the first preset point and the coordinate of the origin of the camera coordinate system; determining a fifth vector according to the coordinate representation of the second preset point and the coordinate of the origin of the camera coordinate system; and determining the first equation set by calculating the magnitudes of the first vector, the second vector, the third vector, the fourth vector, and the fifth vector.

[0008] In some embodiments, the first vector is perpendicular to the second vector, and the positioning equation set further includes a first positioning equation. The method for determining the first positioning equation includes: determining the first positioning equation according to the dot product of the first vector and the second vector.

[0009] In some embodiments, the positioning equation set further includes a second equation set. The method for determining the second equation set includes: determining a sixth vector according to the coordinate representation of the center of the circle and the coordinate representation of the first preset point; determining a seventh vector according to the coordinate representation of the center of the circle and the coordinate representation of the second preset point; and determining the second equation set according to the sixth vector and the seventh vector.

[0010] In some embodiments, the target image further includes a first connection line between the center of the circle and the first target point, and a second connection line between the center of the circle and the second target point. The positioning equation set includes a second positioning equation. The method for determining the second positioning equation includes: identifying a first line segment corresponding to the first connection line in the target image, and identifying a second line segment corresponding to the second connection line in the target image; obtaining a cosine value of an angle between the first line segment and the second line segment in the target image by identifying the angle; and constructing the second positioning equation according to the coordinate representation of the center of the circle, the coordinate representation of the first preset point, the coordinate representation of the second preset point, and the cosine value.

[0011] An embodiment of the present application further provides a spatial positioning device, including: an acquisition module configured to acquire a target image obtained by a camera photographing a marker, where the marker includes a center of a circle, a first target point, and a second target point; an identification module configured to acquire a number of target pixel points occupied by a diameter of the marker in the target image; a calculation module configured to determine a first distance between the center of the circle and an origin of a camera coordinate system according to the number of target pixel points, an internal parameter of the camera, and a pre-stored length parameter; a determination module configured to determine a first preset point located on a connection line between the origin and the first target point, where a distance between the first preset point and the origin is equal to the first distance; the determination module is further configured to determine a second preset point located on a connection line between the origin and the second target point, where a distance between the second preset point and the origin is equal to the first distance; the determination module is further configured to determine coordinate representations of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; the determination module is further configured to construct a positioning equation set according to the first distance, the diameter of the marker, and the coordinate representations; the determination module is further configured to determine coordinates of the center of the circle, the first target point, and the second target point in the camera coordinate system based on the positioning equation set; and the determination module is further configured to determine a position of the target object to be positioned in the camera coordinate system according to the coordinates.

[0012] An embodiment of the present application further provides an electronic device, including: a memory storing at least one instruction; and a processor configured to execute the instruction stored in the memory to implement the spatial positioning method.

[0013] An embodiment of the present application further provides a computer-readable storage medium storing at least one instruction, where the at least one instruction is executed by a processor in an electronic device to implement the spatial positioning method.

[0014] The embodiment of the present application further provides an attitude control mechanism for a robotic arm. A camera is installed on the robotic arm, and the attitude control mechanism includes the electronic device.

[0015] The embodiment of the present application further provides a method for moving a biochip based on a robotic arm. A monocular camera is installed on the robotic arm, and the biochip is associated with a marker. The method for moving the biochip includes: using the monocular camera to capture an image of the marker to obtain a target image, where the marker includes a center point, a first target point, and a second target point; obtaining the number of target pixel points occupied by the diameter of the marker in the target image; determining a first distance between the center point and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the monocular camera, and a pre-stored length parameter; determining a first preset point on the line connecting the origin and the first target point, where the distance between the first preset point and the origin is equal to the first distance; determining a second preset point on the line connecting the origin and the second target point, where the distance between the second preset point and the origin is equal to the first distance; determining the coordinate representations of the center point, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; constructing a positioning equation set according to the first distance, the diameter of the marker, and the coordinate representations; determining the coordinates of the center point, the first target point, and the second target point in the camera coordinate system based on the positioning equation set; determining the position of the biochip in the camera coordinate system according to the coordinates; and adjusting the attitude of the robotic arm to grasp the biochip based on the position of the biochip in the camera coordinate system.

[0016] It can be seen from the above technical solutions that the embodiment of the present application can take a picture of a circular marker provided on a target object to be positioned, obtain a target image containing the marker, determine the distance between the marker and the camera according to the internal parameters of the camera and the diameter of the marker, and construct a positioning equation according to the distance and the coordinate representations of multiple target points on the marker in the camera coordinate system. By solving the positioning equation, the position of the marker in the camera coordinate system is obtained, and then the position of the target object to be positioned in the camera coordinate system is determined. The spatial positioning method provided by the present application can complete the spatial positioning of the target object to be positioned only through computer vision assistance without deploying high-cost positioning devices such as lidar, which can reduce the cost of positioning and improve the efficiency of spatial positioning. In addition, the circular marker can avoid distortion caused by pose changes, thereby improving the accuracy of spatial positioning. Description of the Drawings

[0017] Figure 1 It is a schematic diagram of an application scenario of the spatial positioning method provided by an embodiment of the present application.

[0018] Figure 2 It is a flowchart of a spatial positioning method provided by an embodiment of the present application.

[0019] Figure 3 It is a flowchart of a method for determining a control intention provided by an embodiment of the present application.

[0020] Figure 4 It is a flowchart of a method for determining a target control instruction provided by an embodiment of the present application.

[0021] Figure 5 It is a schematic diagram of an identifier in a camera coordinate system provided by an embodiment of the present application.

[0022] Figure 6 It is a flowchart of a method for determining a first system of equations provided by an embodiment of the present application.

[0023] Figure 7 It is a flowchart of a method for determining a second system of equations provided by an embodiment of the present application.

[0024] Figure 8 It is a schematic diagram of the positional relationship between a first preset point and a second preset point and an identifier provided by an embodiment of the present application.

[0025] Figure 9 It is a flowchart of a method for determining a second positioning equation provided by an embodiment of the present application.

[0026] Figure 10 It is a functional module diagram of a spatial positioning device provided by an embodiment of the present application.

[0027] Figure 11 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0028] Figure 12 It is a robotic arm attitude control mechanism provided by an embodiment of the present application.

[0029] Figure 13 It is a flowchart of a method for moving a biochip based on a robotic arm provided by an embodiment of the present application. Detailed implementation manners

[0030] In order to more clearly understand the purpose, features, and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments may be combined with each other. Many specific details are set forth in the following description in order to fully understand the present application. The described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0031] In addition, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of this application, "a plurality of" means two or more unless otherwise specifically defined.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. The terms used in the description of this application herein are for the purpose of describing specific embodiments only and are not intended to limit this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0033] Embodiments of this application provide a spatial positioning method, which can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0034] An electronic device can be any electronic product that can perform human-computer interaction with a customer. For example, a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an Internet protocol television (IPTV), a smart wearable device, etc.

[0035] An electronic device may also include a network device and / or a client device. Among them, the network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

[0036] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0037] Such as Figure 1As shown, the spatial positioning method provided by this application can be applied to an electronic device 100, and the electronic device 100 is communicatively connected to a camera 200. Among them, the camera 200 receives a shooting instruction sent by the electronic device 100, and shoots a target object 400 with a marker 300 set thereon according to the shooting instruction to obtain a target image including the marker 300.

[0038] In an embodiment of this application, the electronic device 100 can be a processor in the camera 200, or can also be a device with data processing functions communicatively connected to the camera 200, and this application embodiment does not make any limitations in this regard.

[0039] Exemplarily, when the spatial positioning method is applied to an industrial production line, the camera 200 can be a camera arranged on a robotic arm of the production line, the target object 400 to be positioned can be a jig to be grabbed on the production line, and the marker 300 can be a circular sticker pasted on the jig; when the spatial positioning method is applied to an engineering construction scenario, the camera 200 can be a camera arranged at the end of a tower crane, the target object 400 to be positioned can be building materials, and the marker 300 can be any circular mark arranged on the building materials (for example, a circular sticker pasted on the building materials, a circular iron sheet inlaid on the building materials).

[0040] In an embodiment of this application, a first target point (not shown in the figure) and a second target point (not shown in the figure) are arranged on the marker 300. When performing spatial positioning on the target object 400 to be positioned, the marker 300 can be shot by the camera 200 to obtain a target image, and a first distance between the marker 300 and the camera 200 is determined according to the image information corresponding to the marker 300 in the target image and the internal parameters of the camera 200. A positioning equation set is constructed according to the first distance, the coordinate representation of the center of the marker in the camera coordinate system, the coordinate representation of the first target point in the camera coordinate system, and the coordinate representation of the second target point in the camera coordinate system. By solving the positioning equation set, each of the above coordinate representations is obtained, the position of the marker in the camera coordinate system is determined, and further the position of the target object to be positioned in the camera coordinate system is determined.

[0041] As Figure 2 shown, it is a flowchart of the spatial positioning method provided by an embodiment of this application. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted. The spatial positioning method provided by the embodiment of this application includes the following steps.

[0042] S20, obtain a target image obtained by the camera shooting the marker, where the marker includes a center, a first target point, and a second target point.

[0043] In an embodiment of the present application, in order to locate the object to be located using computer vision technology, first, a target image obtained by a camera photographing a marker can be acquired. Since the marker is provided on the object to be located, after the marker is located, the position of the object to be located can be characterized according to the position of the marker.

[0044] In an embodiment of the present application, the marker is circular. For example, the marker can be a circular sticker, or an iron sheet embedded on the surface of the object to be located, etc. The specific type of the marker in the embodiment of the present application is not limited. Among them, the marker includes a center of a circle, a first target point, and a second target point. The distance between the first target point and the center of the circle is equal to the distance between the second target point and the center of the circle, and the first connection line between the first target point and the center of the circle is perpendicular to the second connection line between the second target point and the center of the circle. Exemplarily, when the marker is a circular sticker, the center of the circle is the center of the sticker, and the first target point and the second target point can be pre-set points on the edge of the sticker.

[0045] S21. Obtain the number of target pixel points occupied by the diameter of the marker in the target image.

[0046] In an embodiment of the present application, in order to determine the position of the marker in the camera coordinate system, the number of target pixel points occupied by the diameter of the marker in the target object image can be confirmed first, and then the distance between the center of the target object and the origin of the camera coordinate system can be determined according to the number of target pixel points and the camera internal parameters.

[0047] In an embodiment of the present application, obtaining the number of target pixel points occupied by the diameter of the marker in the target image includes: using a preset target detection algorithm to determine the pixel points occupied by the marker in the target image; determining the widest part among the pixel points as the target pixel points. Among them, the preset target detection algorithm can be an algorithm with target detection functions such as region growing method, watershed algorithm, VGGNet, residual neural network, etc. The specific type of the target detection algorithm in the embodiment of the present application is not limited.

[0048] In an embodiment of the present application, after determining the pixel points of the marker in the target image, all pixel points at the widest part are queried among the pixel points, so as to determine the target pixel points corresponding to the diameter of the marker in the target image.

[0049] S22. Determine a first distance between the center of the circle and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the camera, and a pre-stored length parameter.

[0050] In an embodiment of the present application, the pre-stored length parameter is used to represent the distance in the camera coordinate system represented by each pixel point in the target image when the camera captures the target object at this focal length. To obtain the pre-stored length parameter, the identification object can be preset at the test position in advance, and the test image obtained by the camera capturing the identification object at the test position is acquired. Here, the test position is at a preset test distance from the origin of the camera coordinate system, and the pre-stored length parameter is determined according to the test distance, the camera internal parameters, and the number of test pixel points occupied by the diameter of the identification object in the test image. Specifically, for the method of determining the pre-stored length parameter, please refer to Figure 3 the corresponding description.

[0051] In an embodiment of the present application, to determine the position of the identification object in the camera coordinate system, the first distance between the center of the identification object and the origin of the camera coordinate system can be determined according to the number of target pixel points, the internal parameters of the camera, and the pre-stored length parameter. Among them, the internal parameters of the camera can be the focal length when the camera captures the target image. Specifically, for the method of determining the first distance, please refer to Figure 4 the corresponding description.

[0052] S23. Determine a first preset point on the line connecting the origin and the first target point, where the distance between the first preset point and the origin is equal to the first distance.

[0053] In an embodiment of the present application, to determine the position of the identification object in the camera coordinate system, a first preset point can be determined in the camera coordinate system. This first preset point is on the line connecting the origin of the camera coordinate system and the first target point, and the distance between this first preset point and the origin is equal to the first distance. Subsequently, the position of the identification object in the camera coordinate system can be determined according to the positional relationship between the first preset point, the center of the circle, the origin of the camera coordinate system, and the first target point.

[0054] S24. Determine a second preset point on the line connecting the origin and the second target point, where the distance between the second preset point and the origin is equal to the first distance.

[0055] In an embodiment of the present application, to determine the position of the identification object in the camera coordinate system, a second preset point can also be determined in the camera coordinate system. This second preset point is on the line connecting the origin of the camera coordinate system and the second target point, and the distance between this second preset point and the origin is also equal to the first distance. Subsequently, the position of the identification object in the camera coordinate system can be determined according to the positional relationship between the second preset point, the center of the circle, the origin of the camera coordinate system, and the second target point.

[0056] S25. Determine the coordinate representations of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system.

[0057] In one embodiment of the present application, in order to determine the position of the marker in the camera coordinate system according to the relative position relationship between the center of the circle, the first target point, the second target point, the first preset point and the second preset point, the coordinate representation of the center of the circle, the first target point, the second target point, the first preset point and the second preset point of the marker in the camera coordinate system can be determined first.

[0058] In one embodiment of the present application, the coordinates of the origin O of the camera coordinate system can be represented as: (0,0,0); the coordinates of the center A can be represented as (u,v,w); the coordinates of the first target point B can be represented as (l,m,n); the coordinates of the second target point C can be represented as (i,j,k).

[0059] In one embodiment of the present application, since the first preset point is on the line connecting the origin and the first target point, and the second preset point is on the line connecting the origin and the second target point, the coordinate representation of the first preset point can be determined based on the coordinate representation of the first target point, and the coordinate representation of the second preset point can be determined based on the coordinate representation of the second target point. Therefore, the coordinate representation of the first preset point B' can be (l*t, m*t, n*t), where t represents the preset first proportional variable; the coordinate representation of the second preset point C' can be (i*T, j*T, k*T), where T represents the preset second proportional variable.

[0060] In one embodiment of the present application, the coordinates of the center of the circle, the first target point, and the second target point in the camera coordinate system can be determined based on the above coordinate representation and the relative position relationship between the various points, and then the position of the marker in the camera coordinate system can be determined based on the center of the circle, the first target point, and the second target point.

[0061] S26, constructing a positioning equation group according to the first distance, the diameter of the marker and the coordinate representation.

[0062] In one embodiment of the present application, in order to obtain the coordinates of the center of the circle, the first target point, and the second target point in the camera coordinate system to determine the position of the marker in the camera coordinate system, a positioning equation group can be constructed based on the first distance, the diameter of the marker, and all coordinate representations. Among them, the positioning equation group is used to characterize the relative position relationship between the center of the circle, the first target point, the second target point, the first preset point, and the second preset point. For specific methods for determining the positioning equation group, please refer to Figure 3 、 Figure 4 、 Figure 6 、 Figure 7 and Figure 9 Corresponding instructions.

[0063] S27, based on the positioning equation group, determine the coordinates of the center of the circle, the first target point and the second target point in the camera coordinate system.

[0064] In an embodiment of the present application, the positioning equation set is used to characterize the relative position relationship between the coordinate representations of each point. According to the known distance from the center of the marker to the origin of the camera coordinate system, the positioning equation set is solved to obtain the positions of the center, the first target point, and the second target point in the camera coordinate system.

[0065] S28. Determine the position of the target object to be located in the camera coordinate system according to the coordinates.

[0066] In an embodiment of the present application, since a plane can be determined by three points, after obtaining the coordinates of the center, the first target point, and the second target point, the position and attitude of the marker in the camera coordinate system can be determined according to the above coordinates, and then the position and attitude of the target object to be located in the camera coordinate system can be determined.

[0067] It can be seen from the above technical solutions that the embodiment of the present application can take a photo of the circular marker provided on the target object to be located to obtain a target image including the marker, determine the distance between the marker and the camera according to the internal parameters of the camera and the diameter of the marker, and construct a positioning equation according to the distance and the coordinate representations of multiple target points on the marker in the camera coordinate system. By solving the positioning equation, the position of the marker in the camera coordinate system is obtained, and then the position of the target object to be located in the camera coordinate system is determined. The spatial positioning method provided by the present application can complete the spatial positioning of the target object to be located only through computer vision assistance without deploying high-cost positioning devices such as lidar, which can reduce the cost of positioning and improve the efficiency of spatial positioning. In addition, the circular marker can avoid the distortion caused by the pose change, thereby improving the accuracy of spatial positioning.

[0068] As Figure 3 shown, it is a flowchart of a method for determining a pre-stored length parameter provided by an embodiment of the present application. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted. The method for determining the pre-stored length parameter provided by the embodiment of the present application includes the following steps.

[0069] S30. Obtain a test image captured by the camera of the marker, where the distance between the center of the marker and the origin of the camera coordinate system is equal to the test distance.

[0070] In an embodiment of the present application, in order to determine the pre-stored length parameter, first, a test image captured by the camera of the marker at the test position can be obtained. Among them, the distance between the test position where the marker is located and the origin of the camera coordinate system is equal to the test distance, that is, the distance between the center of the marker and the origin of the camera coordinate system is equal to the test distance.

[0071] S31. Determine the number of test pixel points occupied by the diameter of the identification object in the test image.

[0072] In an embodiment of the present application, determining the number of test pixel points occupied by the diameter of the identification object in the test image includes: using a preset target detection algorithm to determine the pixel points occupied by the identification object in the test image; determining the widest part from the pixel points as the test pixel points. Among them, the preset target detection algorithm can be an algorithm with target detection functions such as region growing method, watershed algorithm, VGGNet, residual neural network, etc. The specific category of the target detection algorithm is not limited in the embodiments of the present application.

[0073] S32. Calculate the length represented by each pixel point in the test image in the camera coordinate system according to the internal parameters of the camera, the test distance, and the number of test pixel points, to obtain the pre-stored length parameter.

[0074] In an embodiment of the present application, the internal parameters of the camera refer to the focal length used when the camera takes pictures of the identification object at the test position. The method for determining the pre-stored length parameter satisfies the following relational expression:

[0075]

[0076] where, l represents the pre-stored length parameter; d represents the diameter of the identification object; f 测试 represents the focal length used when the camera takes pictures of the representation object at the test position; p 测试 represents the number of test pixel points; r 测试 represents the test distance between the center of the identification object and the origin of the camera coordinate system.

[0077] As Figure 4 shown, it is a flowchart of a method for determining the first distance provided by an embodiment of the present application. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted. The method for determining the first distance provided by the embodiments of the present application includes the following steps.

[0078] S40. Calculate the product of the pixel point length parameter and the number of target pixel points to obtain the virtual diameter of the identification object in the target image.

[0079] In an embodiment of the present application, the virtual diameter of the identification object in the target image is obtained by calculating the product between the length parameter and the number of target pixel points. Among them, the virtual diameter is used to characterize the length corresponding in the camera coordinate system of the pixel points occupied by the diameter of the identification object in the target image.

[0080] Exemplarily, as Figure 5As shown, it is a schematic diagram corresponding to the marker mapped to the target image provided by an embodiment of the present application. Among them, the image information corresponding to the marker 51 in the target image is a circle 52. The line segment 54 between the center of the marker 51 and the origin 53 of the camera coordinate system is the distance from the center of the marker to the origin 53 of the camera coordinate system. The line segment 55 between the center of the circle 52 and the origin 53 of the camera coordinate system is the focal length of the camera (hereinafter referred to as the focal length 55). The diameter of the marker 51 is the virtual line segment 56, and the diameter of the circle 52 is the virtual line segment 57. The virtual line segment 57 is the virtual diameter corresponding to the marker 51 in the target image.

[0081] S41. Calculate the ratio between the diameter and the virtual diameter.

[0082] In an embodiment of the present application, the ratio between the distance from the center of the marker to the origin of the camera coordinate system and the internal parameter (focal length) of the camera is equal to the ratio between the diameter of the marker and the virtual diameter. Therefore, the ratio between the diameter of the marker and the virtual diameter can be calculated first, and then the first distance between the center of the marker and the origin of the camera coordinate system can be determined according to this ratio and the camera focal length.

[0083] Exemplarily, as Figure 5 shown, the ratio between the line segment 54 and the focal length 55 is equal to the ratio between the virtual line segment 56 and the virtual line segment 57. Therefore, the ratio between the virtual line segment 56 and the virtual line segment 57 can be determined first, and then the length of the line segment 54 can be determined according to this ratio and the focal length 55 to obtain the distance between the marker 51 and the origin 53.

[0084] S42. Obtain the first distance by calculating the product of the ratio and the internal parameter of the camera.

[0085] In an embodiment of the present application, the method of obtaining the first distance by calculating the product of the ratio and the internal parameter of the camera satisfies the following relationship:

[0086]

[0087] Among them, r represents the first distance between the center of the marker and the origin of the camera coordinate system; d represents the diameter of the marker; f represents the focal length of the camera; p represents the number of target pixel points; l represents the pre-stored length parameter.

[0088] Exemplarily, when the diameter of the target object is 200 millimeters, the focal length of the camera is 3.5 millimeters, the number of target pixel points is 1000, and the pre-stored length parameter is 0.1 millimeter, the calculation method of the first distance between the center of the marker and the origin of the camera coordinate system is as follows:

[0089]

[0090] AsFigure 6 As shown, it is a flowchart of a method for determining a first system of equations provided by an embodiment of the present application. According to different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted. The method for determining the first system of equations provided by the embodiment of the present application includes the following steps.

[0091] S50. Determine a first vector according to the coordinate representation of the center of the circle and the coordinate representation of the first target point; determine a second vector according to the coordinate representation of the center of the circle and the coordinate representation of the second target point.

[0092] In an embodiment of the present application, the coordinates of the center of the marker are (u, v, w), and the coordinate representation of the first target point B is (l, m, n), then the first vector is [l - u, m - v, n - w]; the coordinate representation of the second target point C is (i, j, k), then the second vector is [i - u, j - v, k - w].

[0093] S51. Determine a third vector according to the coordinate representation of the center of the circle and the coordinates of the origin of the camera coordinate system.

[0094] In an embodiment of the present application, the coordinate representation of the center of the marker A is (u, v, w), then the third vector is [u, v, w].

[0095] S52. Determine a fourth vector according to the coordinate representation of the first preset point and the coordinates of the origin of the camera coordinate system.

[0096] In an embodiment of the present application, the coordinate representation of the first preset point is (l * t, m * t, n * t), then the fourth vector is [l * t, m * t, n * t]. Where t represents a preset first magnification factor.

[0097] S53. Determine a fifth vector according to the coordinate representation of the second preset point and the coordinates of the origin of the camera coordinate system.

[0098] In an embodiment of the present application, the coordinate representation of the second preset point is (i * T, j * T, k * T), then the fifth vector is [i * T, j * T, k * T]. Where T represents a preset second magnification factor.

[0099] S54. Determine the first system of equations by calculating the magnitudes of the first vector, the second vector, the third vector, the fourth vector, and the fifth vector.

[0100] In an embodiment of the present application, the magnitude of the first vector is used to characterize the distance between the first target point and the center of the marker, and this distance is a preset distance. Exemplarily, when the first target point is a point on the edge of the marker, the magnitude of the first vector is the radius of the marker; the magnitude of the second vector is used to characterize the distance between the second target point and the origin of the camera coordinate system.

[0101] In an embodiment of the present application, the magnitude of the third vector is used to represent the distance between the center of the marker and the origin of the camera coordinate system; the magnitude of the fourth vector is used to represent the distance between the first preset point and the origin of the camera coordinate system, and the magnitude of the fifth vector is used to represent the distance between the second preset point and the origin of the camera coordinate system, and the magnitudes of the third vector, the fourth vector, and the fifth vector are all equal.

[0102] In an embodiment of the present application, the calculation methods of the magnitudes of the first vector and the second vector satisfy the following relational expressions:

[0103]

[0104]

[0105] Among them, the magnitudes of the first vector and the second vector are both equal to a preset distance. Exemplarily, when the first target point and the second target point are on the edge of the marker, the magnitudes of the first vector and the second vector are both equal to the radius of the marker.

[0106] In an embodiment of the present application, the calculation methods of the magnitudes of the third vector, the fourth vector, and the fifth vector satisfy the following relational expressions:

[0107]

[0108]

[0109]

[0110] Among them, the magnitudes of the third vector, the fourth vector, and the fifth vector are all the distances between the center of the marker and the origin of the camera coordinate system calculated in the foregoing steps.

[0111] In an embodiment of the present application, since the first vector is used to represent the position of the line connecting the center and the first target point in the camera coordinate system, and the second vector is used to represent the position between the center and the second target point, the first vector and the second vector are perpendicular. The positioning equation set further includes a first positioning equation, and the method for determining the first positioning equation includes: determining the first positioning equation according to the dot product of the first vector and the second vector. Specifically, since the first vector and the second vector are perpendicular, the dot product of the first vector and the second vector is 0, and the method for determining the first positioning equation satisfies the following relational expression:

[0112] 0 = (l - u)o(i - u) + (m - v)o(j - v) + (n - w)o(k - w)

[0113] Such as Figure 7As shown, it is a flowchart of a method for determining a second set of equations provided by an embodiment of the present application. According to different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted. The method for determining the second set of equations provided by the embodiment of the present application includes the following steps.

[0114] S60. Determine a sixth vector according to the coordinate representation of the center of the circle and the coordinate representation of the first preset point.

[0115] In an embodiment of the present application, the sixth vector: [l*t - u, m*t - v, n*t - w] is obtained by subtracting the coordinate representation of the center of the circle and the coordinate representation of the first preset point in the same dimension.

[0116] S61. Determine a seventh vector according to the coordinate representation of the center of the circle and the coordinate representation of the second preset point.

[0117] In an embodiment of the present application, the coordinate representation of the center of the circle is (u, v, w), and the coordinate representation of the second preset point is ((i*T, j*T, k*T), so the seventh vector is: [i*T - u, j*T - v, k*T - w].

[0118] S63. Determine the second set of equations according to the sixth vector and the seventh vector.

[0119] In an embodiment of the present application, the second set of equations can be constructed according to the positions of the sixth vector and the seventh vector in the camera coordinate system. Please refer to Figure 8 , the origin of the camera coordinate system is O, the center of the marker is A, the first target point is B, the second target point is C, the first preset point is B', the second preset point is C', the sixth vector is used to represent the position of the line segment AB' in the camera coordinate system, and the seventh vector is used to represent the position of the line segment AC' in the camera coordinate system. Among them, the distance from the origin O of the camera coordinate system to the line segment AB' is the height of the triangle AOB', that is Figure 8 the line segment 81 shown. Then the area S of the triangle AOB' AOB can be represented as: half of the product of the distance of the line segment AB' and the length of the line segment 81. Specifically, the following relationship is satisfied:

[0120]

[0121] S AOB It can also be represented as: half of the product of the modulus of the third vector, the modulus of the fourth vector, and the cosine value of the included angle between the third vector and the fourth vector. Specifically, the following relationship is satisfied:

[0122]

[0123] And The modulus of the third vector and the modulus of the fourth vector are both the distance r from the center of the circle to the origin.

[0124] Therefore, combining the above two formulas, we can get:

[0125]

[0126] Then

[0127]

[0128] Where r represents the distance from the center of the marker to the origin of the camera coordinate system.

[0129] In one embodiment of the present application, it can also be determined according to the Pythagorean theorem Figure 8 The length of the line segment 81 shown in the figure, specifically, the method for determining the length of the line segment 81 according to the Pythagorean theorem satisfies the following relationship:

[0130]

[0131] Where r represents the distance from the center of the marker to the origin of the camera coordinate system.

[0132] In one embodiment of the present application, it is also possible to determine based on the camera focal length and line segment Figure 8 The length of the line segment 81 shown in the figure, specifically, the method for determining the length of the line segment 81 according to the Pythagorean theorem satisfies the following relationship:

[0133]

[0134] where p*l is used to represent the product of the number of pixels occupied by line segment AB in the target image and the length parameter; f represents the focal length.

[0135] In one embodiment of the present application, it can be understood that the height of the triangle AOC' can also be determined by the above method. Specifically, the height of the triangle AOC' satisfies the following relationship:

[0136]

[0137]

[0138]

[0139] Where r represents; p*l is used to represent the product of the number of pixels occupied by the line segment AC in the target image and the length parameter; f represents the focal length.

[0140] In one embodiment of the present application, the target image further includes a first line between the center of the circle and the first target point, and a second line between the center of the circle and the second target point, and the positioning equation group further includes a second positioning equation.Figure 9 As shown, it is a flowchart of a method for determining a second positioning equation provided by an embodiment of the present application. According to different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted. The method for determining the second positioning equation provided by the embodiment of the present application includes the following steps.

[0141] S70, identify the first line segment corresponding to the first connection line in the target image, and identify the second line segment corresponding to the second connection line in the target image.

[0142] In an embodiment of the present application, a target recognition model pre-trained to a converged state can be used to identify the first line segment corresponding to the first connection line in the target image and the second line segment corresponding to the second connection line in the target image. Among them, the target recognition model can be VGGNet.

[0143] S71, obtain the cosine value of the included angle by identifying the included angle between the first line segment and the second line segment in the target image.

[0144] In an embodiment of the present application, a pre-written software package can be used to identify the cosine value of the included angle between the first line segment and the second line segment. For example, the software package can be OpenCV.

[0145] In an embodiment of the present application, the cosine value of the included angle between the first line segment and the second line segment can be denoted as COS(B′AC′).

[0146] S72, construct the second positioning equation according to the coordinate representation of the center of the circle, the coordinate representation of the first preset point, the coordinate representation of the second preset point, and the cosine value.

[0147] In an embodiment of the present application, since the first line segment is the line segment corresponding to the first connection line in the target image, and the first connection line is the connection line between the center of the circle and the first target point, the coordinate representation of the first line segment in the camera coordinate system is the first vector [l - u, m - v, n - w]; the second line segment is the line segment corresponding to the second connection line in the target image, then the coordinate representation of the second line segment in the camera coordinate system is the second vector [i - u, j - v, k - w].

[0148] In an embodiment of the present application, the second positioning equation satisfies the following relational expression:

[0149]

[0150] Please refer to Figure 10 , Figure 10It is a functional module diagram of a spatial positioning device provided by an embodiment of the present application. The spatial positioning device 81 includes an acquisition module 810, an identification module 811, a calculation module 812, and a determination module 813. The module / unit referred to in the present application means a series of computer-readable instruction segments that can be executed by a processor 13 and can complete fixed functions, and are stored in a memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0151] The acquisition module 810 is configured to acquire a target image obtained by a camera photographing a marker, where the marker includes a center point, a first target point, and a second target point;

[0152] The identification module 811 is configured to acquire the number of target pixel points occupied by the diameter of the marker in the target image;

[0153] The calculation module 812 is configured to determine a first distance between the center point and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the camera, and a pre-stored length parameter;

[0154] The determination module 813 is configured to determine a first preset point located on the line connecting the origin and the first target point, where the distance between the first preset point and the origin is equal to the first distance;

[0155] The determination module 813 is further configured to determine a second preset point located on the line connecting the origin and the second target point, where the distance between the second preset point and the origin is equal to the first distance;

[0156] The determination module 813 is further configured to determine the coordinate representations of the center point, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system;

[0157] The determination module 813 is further configured to construct a positioning equation set according to the first distance, the diameter of the marker, and the coordinate representations;

[0158] The determination module 813 is further configured to determine the coordinates of the center point, the first target point, and the second target point in the camera coordinate system based on the positioning equation set;

[0159] The determination module 813 is further configured to determine the position of the target object to be positioned in the camera coordinate system according to the coordinates.

[0160] In an embodiment of the present application, the determining module 813 is further configured to: obtain a test image captured by the camera for the identification object, where the distance between the center of the identification object and the origin of the camera coordinate system is equal to the test distance; determine the number of test pixel points occupied by the diameter of the identification object in the test image; calculate the length represented by each pixel point in the test image in the camera coordinate system according to the internal parameters of the camera, the test distance, and the number of test pixel points, so as to obtain the pre-stored length parameter.

[0161] In an embodiment of the present application, the determining module 813 is specifically configured to: calculate the product of the pixel point length parameter and the number of target pixel points to obtain the virtual diameter of the identification object in the target image; calculate the ratio between the diameter and the virtual diameter; obtain the first distance by calculating the product of the ratio and the internal parameters of the camera.

[0162] In an embodiment of the present application, the positioning equation set includes a first equation set, and the determining module 813 is specifically configured to: determine a first vector according to the coordinate representation of the center of the circle and the coordinate representation of the first target point; determine a second vector according to the coordinate representation of the center of the circle and the coordinate representation of the second target point; determine a third vector according to the coordinate representation of the center of the circle and the coordinates of the origin of the camera coordinate system; determine a fourth vector according to the coordinate representation of the first preset point and the coordinates of the origin of the camera coordinate system; determine a fifth vector according to the coordinate representation of the second preset point and the coordinates of the origin of the camera coordinate system; determine the first equation set by calculating the magnitudes of the first vector, the second vector, the third vector, the fourth vector, and the fifth vector.

[0163] In an embodiment of the present application, the first vector is perpendicular to the second vector, and the positioning equation set further includes a first positioning equation. The determining module 813 is further configured to: determine the first positioning equation according to the dot product of the first vector and the second vector.

[0164] In an embodiment of the present application, the positioning equation set further includes a second equation set, and the determining module 813 is specifically configured to: determine a sixth vector according to the coordinate representation of the center of the circle and the coordinate representation of the first preset point; determine a seventh vector according to the coordinate representation of the center of the circle and the coordinate representation of the second preset point; determine the second equation set according to the sixth vector and the seventh vector.

[0165] In an embodiment of the present application, the target image further includes a first connection line between the center of the circle and the first target point, and a second connection line between the center of the circle and the second target point. The positioning equation set includes a second positioning equation. The determining module 813 is specifically configured to: identify a first line segment corresponding to the first connection line in the target image, and identify a second line segment corresponding to the second connection line in the target image; obtain a cosine value of an angle between the first line segment and the second line segment in the target image by identifying the angle; and construct the second positioning equation according to the coordinate representation of the center of the circle, the coordinate representation of the first preset point, the coordinate representation of the second preset point, and the cosine value.

[0166] As can be seen from the above technical solutions, the embodiments of the present application can take a photo of a circular marker disposed on a target object to be positioned to obtain a target image including the marker, determine the distance between the marker and the camera according to the camera internal parameters and the diameter of the marker, and construct a positioning equation according to the distance and the coordinate representations of multiple target points on the marker in the camera coordinate system. By solving the positioning equation, the position of the marker in the camera coordinate system can be obtained, and then the position of the target object to be positioned in the camera coordinate system can be determined. The spatial positioning method provided by the present application can complete the spatial positioning of the target object to be positioned only through computer vision assistance without deploying high-cost positioning devices such as lidar, which can reduce the positioning cost and improve the efficiency of spatial positioning. In addition, the circular marker can avoid distortion caused by pose changes, thereby improving the accuracy of spatial positioning.

[0167] Please refer to Figure 11 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 is used to execute the computer-readable instructions stored in the memory to implement the spatial positioning method described in any of the above embodiments.

[0168] In an embodiment of the present application, the electronic device 1 further includes a bus and a computer program stored in the memory 12 and executable on the processor 13, such as a spatial positioning program.

[0169] Figure 11 Only the electronic device 1 having the memory 12 and the processor 13 is shown. Those skilled in the art can understand that Figure 11 the shown structure does not constitute a limitation on the electronic device 1, and it may include fewer or more components than shown, or combine some components, or have different component arrangements.

[0170] In combination with Figure 2, the memory 12 in the electronic device 1 stores multiple computer-readable instructions to implement a spatial positioning method. The processor 13 can execute the multiple instructions to implement: obtaining a target image captured by the camera of the identification object, where the identification object includes a center point, a first target point, and a second target point; obtaining the number of target pixel points occupied by the diameter of the identification object in the target image; determining a first distance between the center point and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the camera, and a pre-stored length parameter; determining a first preset point on the line connecting the origin and the first target point, where the distance between the first preset point and the origin is equal to the first distance; determining a second preset point on the line connecting the origin and the second target point, where the distance between the second preset point and the origin is equal to the first distance; determining the coordinate representations of the center point, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; constructing a positioning equation set according to the first distance, the diameter of the identification object, and the coordinate representations; determining the coordinates of the center point, the first target point, and the second target point in the camera coordinate system based on the positioning equation set; and determining the position of the object to be located in the camera coordinate system according to the coordinates.

[0171] Specifically, for the specific implementation method of the above instructions by the processor 13, reference can be made to Figure 2 the description of the relevant steps in the corresponding embodiment, which will not be elaborated here.

[0172] Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 can be a bus structure or a star structure. The electronic device 1 can also include more or fewer other hardware or software than shown in the figure, or different component arrangements. For example, the electronic device 1 can also include input / output devices, network access devices, etc.

[0173] It should be noted that the electronic device 1 is only an example. Other existing or future electronic products that can be adapted to this application should also be included in the protection scope of this application and are included herein by reference.

[0174] Among them, the memory 12 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as the mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 1. The memory 12 can be used not only to store application software installed in the electronic device 1 and various types of data, such as the code of the space positioning program, etc., but also to temporarily store data that has been output or will be output.

[0175] In some embodiments, the processor 13 can be composed of integrated circuits. For example, it can be composed of a single packaged integrated circuit, or can be composed of multiple integrated circuits with the same or different functions packaged, including a combination of one or more Central Processing Units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips, etc. The processor 13 is the control core (Control Unit) of the electronic device 1, connecting various components of the entire electronic device 1 through various interfaces and circuits. By running or executing programs or modules stored in the memory 12 (such as executing the space positioning program, etc.), and calling data stored in the memory 12, it performs various functions of the electronic device 1 and processes data.

[0176] The processor 13 executes the operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in the above-mentioned embodiments of each space positioning method, for example Figure 2 the steps shown.

[0177] Exemplarily, the computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory 12 and executed by the processor 13 to complete this application. The one or more modules / units can be a series of computer-readable instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program can be divided into an acquisition module 810, an identification module 811, a calculation module 812, and a determination module 813.

[0178] The integrated units implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above-mentioned software functional modules stored in a storage medium include several instructions for causing a computer device (which may be a personal computer, a computer device, or a network device, etc.) or a processor to execute a part of the spatial positioning method described in each embodiment of the present application.

[0179] If the integrated module / unit of the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present application, it can also be completed by a computer program instructing relevant hardware devices. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented.

[0180] Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory, and other memories, etc.

[0181] Furthermore, the computer-readable storage medium mainly includes a storage program area and a storage data area. Among them, the storage program area can store an operating system, application programs required for at least one function, etc.; the storage data area can store data created according to the use of the blockchain node, etc.

[0182] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, in Figure 8 only one arrow is used to represent it, but it does not mean that there is only one bus or one type of bus. The bus is set to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0183] An embodiment of the present application further provides a computer-readable storage medium (not shown in the figure), in which computer-readable instructions are stored, and the computer-readable instructions are executed by a processor in an electronic device to implement the spatial positioning method described in any of the above embodiments.

[0184] An embodiment of the present application further provides an attitude control mechanism for a robotic arm. As Figure 12 shown, a camera 600 is installed on the robotic arm 500, and the attitude control mechanism 700 is connected to the robotic arm 500 for controlling the attitude of the robotic arm 500. Among them, the attitude control mechanism 700 includes an electronic device 1 as Figure 11 shown.

[0185] The present application also provides a method for moving a biochip based on a robotic arm. A monocular camera is installed on the robotic arm, and the biochip is associated with a marker. As Figure 13 shown, it is a flowchart of a method for moving a biochip based on a robotic arm provided by an embodiment of the present application. According to different requirements, the order of steps in this flowchart can be changed, and some steps can be omitted. The method for moving a biochip based on a robotic arm provided by an embodiment of the present application includes the following steps.

[0186] S80, using the monocular camera to capture the marker to obtain a target image, where the marker includes a center point, a first target point, and a second target point.

[0187] In an embodiment of the present application, the method of using the monocular camera to capture the marker to obtain a target image is the same as the detailed description in step S20, and will not be repeated here.

[0188] S81, obtaining the number of target pixel points occupied by the diameter of the marker in the target image.

[0189] In an embodiment of the present application, the method of obtaining the number of target pixel points occupied by the diameter of the marker in the target image is the same as the detailed description in step S21, and will not be repeated here.

[0190] S82, determining a first distance between the center point and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the monocular camera, and a pre-stored length parameter.

[0191] In an embodiment of the present application, the method of determining a first distance between the center point and the origin of the camera coordinate system according to the number of target pixel points, the internal parameters of the monocular camera, and a pre-stored length parameter is the same as the detailed description in step S22, and will not be repeated here.

[0192] S83. Determine a first preset point on the line connecting the origin and the first target point, where the distance between the first preset point and the origin is equal to the first distance.

[0193] In an embodiment of the present application, the method for determining the first preset point on the line connecting the origin and the first target point is the same as the detailed description in step S23, and will not be elaborated here.

[0194] S84. Determine a second preset point on the line connecting the origin and the second target point, where the distance between the second preset point and the origin is equal to the first distance.

[0195] In an embodiment of the present application, the determination of the second preset point on the line connecting the origin and the second target point is the same as the detailed description in step S24, and will not be elaborated here.

[0196] S85. Determine the coordinate representations of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system.

[0197] In an embodiment of the present application, the determination of the coordinate representations of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system is the same as the detailed description in step S25, and will not be elaborated here.

[0198] S86. Construct a positioning equation system based on the first distance, the diameter of the marker, and the coordinate representations.

[0199] In an embodiment of the present application, the construction of the positioning equation system based on the first distance, the diameter of the marker, and the coordinate representations is the same as the detailed description in step S26, and will not be elaborated here.

[0200] S87. Based on the positioning equation system, determine the coordinates of the center of the circle, the first target point, and the second target point in the camera coordinate system.

[0201] In an embodiment of the present application, the determination of the coordinates of the center of the circle, the first target point, and the second target point in the camera coordinate system based on the positioning equation system is the same as the detailed description in step S27, and will not be elaborated here.

[0202] S88. Determine the position of the biochip in the camera coordinate system according to the coordinates.

[0203] In an embodiment of the present application, the determination of the position of the biochip in the camera coordinate system according to the coordinates is the same as the detailed description in step S28, and will not be elaborated here.

[0204] S89. Adjust the posture of the robotic arm to grasp the biochip based on the position of the biochip in the camera coordinate system.

[0205] In an embodiment of the present application, after determining the position of the biochip in the camera coordinate system, the posture of the robotic arm can be adjusted according to the position of the biochip in the camera coordinate system so that the robotic arm grasps the biochip.

[0206] For example, when the position of the biochip in the camera coordinate system is (x, y, z), the point position information of the robotic arm can be constructed according to this coordinate using a pre-stored point position construction program. This point position information is used to characterize the posture of the robotic arm. Among them, the first three dimensions are used to characterize the position of the gripper of the robotic arm in the camera coordinate system, and the last three dimensions are used to characterize the angles between the gripper and the X-axis, Y-axis, and Z-axis. In an embodiment of the present application, the electronic device in the robotic arm posture control mechanism receives this point position information and adjusts the posture of the robotic arm according to this point position information so that the robotic arm grasps the biochip.

[0207] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation.

[0208] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, in each embodiment of the present application, the various functional modules can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of a combination of hardware and software functional modules.

[0210] In addition, obviously, the term "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described in the specification can also be implemented by one unit or device through software or hardware. The terms such as the first and the second are used to represent names and do not represent any specific order.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A spatial positioning method, applied to electronic equipment, characterized in that: The electronic device is in communication connection with a camera, the camera is used to photograph a target object to be located, wherein a marker is provided on the target object to be located, and the marker is circular, and the method comprises: Acquire a target image obtained by photographing the marker by the camera, wherein the marker includes a center of a circle, a first target point, and a second target point; Obtain the number of target pixel points occupied by the diameter of the marker in the target image; Determine a first distance between the center of the circle and the origin of the camera coordinate system according to the number of target pixels, the intrinsic parameters of the camera, and a pre-stored length parameter; Determine a first preset point located on a line connecting the origin and the first target point, wherein a distance between the first preset point and the origin is equal to the first distance; Determine a second preset point located on a line connecting the origin and the second target point, wherein a distance between the second preset point and the origin is equal to the first distance; Determine coordinate representations of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; Constructing a positioning equation group according to the first distance, the diameter of the marker and the coordinate representation; Based on the positioning equation group, determining the coordinates of the center of the circle, the first target point and the second target point in the camera coordinate system; The position of the target object to be located in the camera coordinate system is determined according to the coordinates.

2. The spatial positioning method according to claim 1, characterized in that: The method for determining the pre-stored pixel length parameter includes: Acquire a test image obtained by photographing the marker by the camera, wherein the distance between the center of the marker and the origin of the camera coordinate system is equal to the test distance; Determine the number of test pixels occupied by the diameter of the marker in the test image; The length represented by each pixel in the test image in the camera coordinate system is calculated according to the intrinsic parameters of the camera, the test distance and the number of test pixels to obtain the pre-stored length parameter.

3. The spatial positioning method according to claim 1, characterized in that: Determining the first distance between the center of the circle and the origin of the camera coordinate system according to the number of target pixels, the intrinsic parameters of the camera, and the pre-stored pixel length parameters includes: Calculate the product of the pixel length parameter and the number of target pixels to obtain a virtual diameter of the marker in the target image; calculating a ratio between the diameter and the virtual diameter; The first distance is obtained by calculating the product of the ratio and the camera intrinsic parameter.

4. The spatial positioning method according to claim 1, characterized in that: The positioning equation group includes a first equation group, and the method for determining the first equation group includes: Determine a first vector according to the coordinate representation of the center of the circle and the coordinate representation of the first target point; determine a second vector according to the coordinate representation of the center of the circle and the coordinate representation of the second target point; Determine a third vector according to the coordinate representation of the center of the circle and the coordinates of the origin of the camera coordinate system; Determine a fourth vector according to the coordinate representation of the first preset point and the coordinates of the origin of the camera coordinate system; Determine a fifth vector according to the coordinate representation of the second preset point and the coordinates of the origin of the camera coordinate system; The first set of equations is determined by calculating the moduli of the first vector, the second vector, the third vector, the fourth vector, and the fifth vector.

5. The spatial positioning method according to claim 4, characterized in that: The first vector is perpendicular to the second vector, the positioning equation group further includes a first positioning equation, and a method for determining the first positioning equation includes: The first positioning equation is determined according to the dot product of the first vector and the second vector.

6. The spatial positioning method according to claim 1, characterized in that: The positioning equation group also includes a second equation group, and the method for determining the second equation group includes: Determine a sixth vector according to the coordinate representation of the center of the circle and the coordinate representation of the first preset point; Determine a seventh vector according to the coordinate representation of the center of the circle and the coordinate representation of the second preset point; The second set of equations is determined based on the sixth vector and the seventh vector.

7. The spatial positioning method according to claim 6, characterized in that: The target image also includes a first line between the center of the circle and the first target point, and a second line between the center of the circle and the second target point. The positioning equation group includes a second positioning equation. The method for determining the second positioning equation includes: Identify a first line segment corresponding to the first connecting line in the target image, and identify a second line segment corresponding to the second connecting line in the target image; By identifying the angle between the first line segment and the second line segment in the target image, a cosine value of the angle is obtained; The second positioning equation is constructed according to the coordinate representation of the center of the circle, the coordinate representation of the first preset point, the coordinate representation of the second preset point and the cosine value.

8. A spatial positioning device, characterized in that: The device comprises a module for implementing the spatial positioning method according to any one of claims 1 to 7, and the device comprises: An acquisition module, used to acquire a target image obtained by photographing a marker by a camera, wherein the marker includes a center of a circle, a first target point, and a second target point; A recognition module, used for obtaining the number of target pixels occupied by the diameter of the marker in the target image; A calculation module, used to determine a first distance between the center of the circle and the origin of the camera coordinate system according to the number of target pixels, the intrinsic parameters of the camera and a pre-stored length parameter; A determination module, configured to determine a first preset point located on a line connecting the origin and the first target point, wherein a distance between the first preset point and the origin is equal to the first distance; The determination module is further used to determine a second preset point located on a line connecting the origin and the second target point, wherein the distance between the second preset point and the origin is equal to the first distance; The determination module is further used to determine the coordinate representation of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; The determination module is further used to construct a positioning equation group according to the first distance, the diameter of the marker and the coordinate representation; The determination module is further used to determine the coordinates of the center of the circle, the first target point and the second target point in the camera coordinate system based on the positioning equation group; The determination module is further used to determine the position of the target object to be located in the camera coordinate system according to the coordinates.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing computer-readable instructions; and A processor executes computer-readable instructions stored in the memory to implement the spatial positioning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the spatial positioning method according to any one of claims 1 to 7 is implemented.

11. A posture control mechanism for a robotic arm, wherein a camera is mounted on the robotic arm, characterized in that: The posture control mechanism comprises the electronic device as claimed in claim 9.

12. A biochip moving method based on a robotic arm, characterized in that: A monocular camera is installed on the robot arm, the biochip is associated with the marker, and the biochip moving method includes: Using the monocular camera to photograph the marker to obtain a target image, wherein the marker includes a circle center, a first target point, and a second target point; Obtain the number of target pixel points occupied by the diameter of the marker in the target image; Determine a first distance between the center of the circle and the origin of the camera coordinate system according to the number of target pixels, an internal parameter of the monocular camera, and a pre-stored length parameter; Determine a first preset point located on a line connecting the origin and the first target point, wherein a distance between the first preset point and the origin is equal to the first distance; Determine a second preset point located on a line connecting the origin and the second target point, wherein a distance between the second preset point and the origin is equal to the first distance; Determine coordinate representations of the center of the circle, the first target point, the second target point, the first preset point, and the second preset point in the camera coordinate system; Constructing a positioning equation group according to the first distance, the diameter of the marker and the coordinate representation; Based on the positioning equation group, determining the coordinates of the center of the circle, the first target point and the second target point in the camera coordinate system; Determining the position of the biochip in the camera coordinate system according to the coordinates; and Based on the position of the biochip in the camera coordinate system, the posture of the robot arm is adjusted to grasp the biochip.

Images