Robotic arm control method, apparatus, system, and readable storage medium

Abstract

The application relates to a mechanical arm control method, device, system and readable storage medium, which comprises the following steps: acquiring an instrument image of a dashboard to be read, and determining an edge line of the dashboard in the instrument image according to the instrument image; the instrument image is obtained by a shooting component arranged on a mechanical arm; determining a projection point of each edge point on the edge line on a preset plane, and determining a plurality of projection vectors according to each edge point and the corresponding projection point; determining a rotation angle according to the plurality of projection vectors, and controlling a rotating shaft of the mechanical arm to rotate according to the rotation angle, and controlling the mechanical arm to shoot the dashboard by using the shooting component after rotation to obtain a target instrument image. The mechanical arm control method provided by the application can obtain a more accurate target instrument image.

Description

Technical Field

[0001] This application relates to the field of control technology, and in particular to a robotic arm control method, apparatus, system, and readable storage medium. Background Technology

[0002] In the grand scheme of smart power plants, robots are gradually becoming an indispensable and important force. With their high efficiency, precision, and safety, they are profoundly changing the traditional operation model and energy efficiency management level of power plants. Regarding power plant safety, robots are used to inspect the plant, capture images of instruments, collect data from each instrument, and automate the reading of instrument readings.

[0003] However, in traditional technologies, the images of instruments obtained by robots are inaccurate. Summary of the Invention

[0004] Therefore, it is necessary to provide a robotic arm control method, device, system, and readable storage medium that can improve the accuracy of the captured instrument images, addressing the aforementioned technical problems.

[0005] In a first aspect, this application provides a robotic arm control method, the method comprising:

[0006] The instrument image of the instrument panel to be read is acquired, and the edge line of the instrument panel in the instrument image is determined based on the instrument image; the instrument image is obtained by capturing images from a camera component mounted on the robotic arm.

[0007] Determine the projection points of each edge point on the edge line onto a preset plane, and determine multiple projection vectors based on each edge point and its corresponding projection point;

[0008] The rotation angle is determined based on multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate according to the rotation angle. After the robotic arm rotates, it uses a camera component to capture an image of the instrument panel to obtain the target instrument image.

[0009] In one embodiment, the projection points of each edge point on the edge line onto a preset plane are determined, and multiple projection vectors are determined based on each edge point and its corresponding projection point; including:

[0010] Determine the center point of the instrument panel based on the instrument image;

[0011] Construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system;

[0012] Based on the first coordinates of each edge point, determine the second coordinates of the projection points of each edge point on the preset plane;

[0013] Multiple projection vectors are determined based on multiple first coordinates and their corresponding second coordinates.

[0014] In one embodiment, multiple projection vectors are determined based on multiple first coordinates and corresponding second coordinates, including:

[0015] For each quadrant in the target coordinate system, multiple first coordinates are used to determine multiple initial projection vectors based on the first coordinates and the corresponding second coordinates.

[0016] Multiple initial projection vectors are arranged in a preset order to obtain multiple projection vectors.

[0017] In one embodiment, the method further includes:

[0018] For each quadrant, a sub-adjustment matrix is ​​determined based on multiple initial projection vectors.

[0019] By sequentially concatenating multiple sub-adjustment matrices, the adjustment matrices corresponding to multiple projection vectors are determined.

[0020] The rotation angle is determined based on multiple projection vectors, including:

[0021] The rotation angle is determined by the adjustment matrix corresponding to multiple projection vectors.

[0022] In one embodiment, determining the rotation angle based on multiple projection vectors includes:

[0023] Multiple projection vectors are input into a pre-trained angle determination model, which then outputs the rotation angle.

[0024] In one embodiment, acquiring an instrument image of the dashboard to be read includes:

[0025] Acquire the initial instrument image of the dashboard and determine the distance between the dashboard and the end effector of the robotic arm based on the initial instrument image;

[0026] Based on the distance and distance threshold, the distance adjustment parameters are determined; and the movement of the robotic arm is controlled according to the distance adjustment parameters. After the robotic arm moves, it uses the imaging component to capture images of the instrument panel to obtain instrument images.

[0027] In one embodiment, the robotic arm, after moving, uses a camera component to capture an image of the dashboard, obtaining an image of the dashboard, including:

[0028] Adjust the focal length of the shooting component based on the distance adjustment parameters;

[0029] After the robotic arm moves, it uses the adjusted camera module to capture images of the instrument panel, thus obtaining an image of the instrument.

[0030] Secondly, one embodiment of this application provides a robotic arm control device, the device comprising:

[0031] The acquisition module is used to acquire the instrument image of the instrument panel to be read, and to determine the edge line of the instrument panel in the instrument image based on the instrument image; the instrument image is obtained by capturing images from a camera component mounted on the robotic arm.

[0032] The determination module is used to determine the projection points of each edge point on the edge line onto a preset plane, and to determine multiple projection vectors based on each edge point and its corresponding projection point.

[0033] The control module is used to determine the rotation angle based on multiple projection vectors, control the rotation axis of the robotic arm to rotate according to the rotation angle, and control the robotic arm to take pictures of the instrument panel using the imaging component after rotation to obtain the target instrument image.

[0034] Thirdly, one embodiment of this application also provides a robotic arm control system, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the first aspect above.

[0035] Fourthly, one embodiment of this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0036] Fifthly, one embodiment of this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect.

[0037] The aforementioned robotic arm control method, device, system, and readable storage medium involve acquiring an instrument image of a dashboard to be read, and determining the edge lines of the dashboard in the instrument image based on the instrument image. The instrument image is captured by a camera component mounted on the robotic arm. The method determines the projection points of each edge point on the edge line onto a preset plane, and determines multiple projection vectors based on each edge point and its corresponding projection point. A rotation angle is determined based on these multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate according to the rotation angle. After rotation, the robotic arm uses the camera component to capture an image of the dashboard to obtain the target instrument image. In this embodiment, the multiple projection vectors formed by each edge point on the dashboard's edge line and its corresponding projection point represent the degree of distortion of the dashboard. By controlling the rotation axis of the robotic arm to rotate according to the rotation angle determined by the multiple projection vectors, the robotic arm can be rotated to the optimal angle. After rotation, the robotic arm uses the camera component to capture an image of the dashboard, thus avoiding distortion of the target instrument image and improving its quality. This, in turn, improves the accuracy of the instrument data read from the target instrument image. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of a computer device in one embodiment;

[0039] Figure 2 This is a flowchart illustrating the steps of a robotic arm control method in one embodiment.

[0040] Figure 3 This is a flowchart illustrating the steps of a robotic arm control method in another embodiment;

[0041] Figure 4 This is a flowchart illustrating the steps of a robotic arm control method in another embodiment;

[0042] Figure 5 This is a flowchart illustrating the steps of a robotic arm control method in another embodiment;

[0043] Figure 6 This is a flowchart illustrating the steps of a robotic arm control method in another embodiment;

[0044] Figure 7 This is a flowchart illustrating the steps of a robotic arm control method in another embodiment;

[0045] Figure 8 This is a flowchart illustrating the steps of a robotic arm control method in another embodiment;

[0046] Figure 9 This is a schematic diagram of the robotic arm control device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] The serial numbers assigned to components in this article, such as "first" and "second", are used only to distinguish the objects being described and have no sequential or technical meaning.

[0049] First, before introducing the technical solutions of the embodiments disclosed in this application, we will first introduce the background technology or the technological evolution on which the embodiments of this application are based. In the grand scenario of smart power plants, robots are gradually becoming an indispensable and important player. With their high efficiency, accuracy, and safety, they are profoundly changing the operation mode and energy efficiency management level of traditional power plants. As the forefront of the digital transformation of the energy industry, smart power plants integrate advanced technologies such as the Internet of Things, big data, and artificial intelligence, and robots are the vivid practice of these technologies in action. Robots in smart power plants also have a high degree of environmental adaptability and learning ability. They can be customized according to the actual operational needs of the power plant, continuously optimize workflows, and improve operational efficiency. Through seamless integration with the power plant management system, the data collected by the robots can be uploaded to the cloud platform in real time, providing data support for the intelligent decision-making of the power plant and helping to achieve more efficient, environmentally friendly, and sustainable energy production. In terms of daily inspections, robots, with their excellent mobility and high-precision sensors, can autonomously perform detailed inspections of various areas of the power plant, including key parts such as generator sets, transmission lines, and cooling systems, effectively replacing manual labor in completing inspection tasks in high-risk or hard-to-reach areas. The robot can not only collect equipment operation data in real time, but also promptly detect potential faults or overheating phenomena through built-in image recognition and thermal imaging analysis technologies, providing strong support for preventive maintenance and significantly reducing equipment failure rates and unplanned downtime. In terms of power plant safety, the robot performs inspections of the power plant and collects instrument data by capturing images of various instruments, completing the automated reading of instrument numbers. However, in traditional technologies, it is difficult to guarantee the optimal shooting angle when the robot captures images of the instruments, leading to inaccurate images and consequently inaccurate instrument numbers obtained from those images. To address this, this application provides a robotic arm control method.

[0050] The robotic arm control method provided in this application can be applied to a robotic arm control system. The robotic arm control system includes a controller, a moving part, a robotic arm mounted on the moving part, and a camera assembly located at the end of the robotic arm. A drive component for movement is located at the bottom of the moving part; the drive component can be a moving leg, a moving wheel, or a moving track. The camera assembly at the end of the robotic arm can be a depth camera. A depth camera is a camera capable of capturing distance information between the object being photographed and the depth camera, and generating a 3D scene. The depth camera utilizes depth sensor and image processor technology, employing different working principles, such as 3D structured light technology, binocular vision technology, or Time of Flight (TOF) technology, to perceive and measure the depth of the object being photographed. Specifically, in 3D structured light technology, the depth camera emits light and receives the reflected signals, determining the distance between the object being photographed and the depth camera by calculating the time difference or phase shift of the light rays.

[0051] The controller can be a computer device, such as Figure 1 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a robotic arm control method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0052] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0053] The technical solution of this application and how the technical solution of this application solves the technical problem are described in detail below with specific embodiments.

[0054] In one embodiment, such as Figure 2As shown, a robotic arm control method is provided. This embodiment illustrates the application of this method to a controller in a robotic arm control system. In this embodiment, the method includes the following steps:

[0055] Step 200: Obtain the instrument image of the instrument panel to be read, and determine the edge line of the instrument panel in the instrument image based on the instrument image; the instrument image is obtained by taking pictures based on the shooting component set on the robotic arm.

[0056] The instrument panel to be read can be the instrument panel of various instruments in the power system. Instruments in the power system can include temperature instruments, pressure instruments, current instruments, flow instruments, electric instruments, etc. The shapes of various instrument panels can be circular, oval, near-circular, square, sector-shaped, etc.

[0057] In the robotic arm control system, after the controller moves the robotic arm to its initial position, it controls the imaging component at the end of the robotic arm to capture an image of the instrument panel to be read. The initial position can be preset by the user or it can be a position reached by the robotic arm during its automatic movement according to a preset program.

[0058] After acquiring the instrument image, the controller can determine the edge lines of the instrument panel by analyzing the image. The edge lines of the instrument panel refer to the lines or outlines surrounding the dial or display screen. This embodiment does not limit the specific method for determining the edge lines of the instrument panel in the instrument image, as long as the function is achieved.

[0059] In an optional embodiment, the controller pre-stores an image analysis model. The controller inputs the acquired instrument image into the image analysis model to obtain the edge lines of the instrument panel in the instrument image. The controller can also perform feature analysis on the instrument image to determine the edge lines of the instrument panel.

[0060] Step 210: Determine the projection points of each edge point on the edge line onto the preset plane, and determine multiple projection vectors based on each edge point and its corresponding projection point.

[0061] The preset plane can be any plane in the coordinate system where the robotic arm is located, or any plane in a coordinate system determined based on the dashboard image. Specifically, the preset plane can be a plane parallel to the ground where the robotic arm is located. This embodiment does not limit the preset plane, as long as it can achieve its function.

[0062] After determining the edge line of the instrument panel in the instrument image, the controller determines each edge point on the edge line. This embodiment does not limit the specific method for determining each edge point on the edge line, nor the number of edge points determined, as long as the function can be achieved.

[0063] In an optional embodiment, the controller can collect data from points on the edge line at preset intervals and define each collected point as an edge point. The preset intervals between the edge points can be the same or different.

[0064] After determining each edge point on the edge line, the controller determines the projection point of each edge point on a preset plane. Based on the edge point and its corresponding projection point, the controller can determine the projection vector corresponding to that edge point, thus obtaining multiple projection vectors for each edge point. The direction of the projection vector can be either from the edge point to the corresponding projection point or from the projection point to the corresponding edge point. The projection vector corresponding to each edge point can represent the degree of distortion of that edge point.

[0065] In an optional embodiment, if the preset plane is parallel to the plane where the instrument panel is located, it indicates that the robotic arm is currently at the optimal shooting angle, and the rotation angle subsequently determined based on multiple projection vectors can be zero. If the preset plane is not parallel to the plane where the instrument panel is located, the degree of distortion of the instrument image can be represented by the determined multiple projection vectors, which facilitates the subsequent determination of the rotation angle and control of the robotic arm's rotation axis.

[0066] Step 220: Determine the rotation angle based on multiple projection vectors, control the rotation axis of the robotic arm to rotate according to the rotation angle, and control the robotic arm to take a picture of the instrument panel using the imaging component after rotation to obtain the target instrument image.

[0067] After determining multiple projection vectors, the controller can determine the rotation angle based on these vectors. The rotation angle determined from the multiple projection vectors can be multiple. The number of rotation angles is the same as the number of rotation axes of the robotic arm. For example, if the robotic arm has three rotation axes, corresponding to rotation in three directions in three-dimensional space, the determined rotation angles include three angles, one for each rotation axis. If the robotic arm has six rotation axes, corresponding to six degrees of freedom in three-dimensional space, these six degrees of freedom include three translational degrees of freedom and three rotational degrees of freedom.

[0068] After the controller controls the rotation axis of the robotic arm to rotate according to the rotation angle, the camera component set on the end of the robotic arm takes a picture of the instrument panel to be read, and the target instrument image of the instrument panel can be obtained.

[0069] In an optional embodiment, after acquiring the target instrument image, the controller can read the data from the dashboard by analyzing the target instrument image. Specifically, the controller can input the target instrument image into a pre-trained data reading model, which can then output the data from the dashboard.

[0070] The robotic arm control method provided in this application acquires an instrument image of a dashboard to be read, and determines the edge lines of the dashboard in the instrument image. The instrument image is captured by a camera component mounted on the robotic arm. The method determines the projection points of each edge point on the edge line onto a preset plane, and determines multiple projection vectors based on each edge point and its corresponding projection point. A rotation angle is determined based on these multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate according to the rotation angle. After rotation, the robotic arm uses the camera component to capture an image of the dashboard to obtain the target instrument image. In this embodiment, the multiple projection vectors formed by each edge point on the dashboard edge line and its corresponding projection point represent the degree of distortion of the dashboard. By controlling the rotation axis of the robotic arm to rotate according to the rotation angle determined by the multiple projection vectors, the robotic arm can be rotated to the optimal angle. After rotation, the robotic arm uses the camera component to capture an image of the dashboard, thus avoiding distortion of the target instrument image and improving its quality. This, in turn, improves the accuracy of the instrument data read from the target instrument image.

[0071] In one embodiment, one approach involves determining a rotation angle based on multiple projection vectors, the approach including:

[0072] Multiple projection vectors are input into a pre-trained angle determination model, which then outputs the rotation angle.

[0073] The angle determination model can be obtained by training a neural network model using training samples and pre-stored in the controller's storage unit. Training samples may include projection vector samples and standard rotation angles. The neural network model can be a convolutional neural network model, or a multilayer perceptron, etc. The type of angle determination model is the same as that of the neural network model. This embodiment does not limit the type or structure of the neural network model, as long as it can achieve its function.

[0074] After acquiring multiple projection vectors, the controller inputs these vectors into the angle determination model. Through analysis and processing of the angle determination model, the rotation angle can be output.

[0075] In this embodiment, the rotation angle can be determined by directly inputting multiple projection vectors into a pre-trained angle determination model. This method of determining the rotation angle is fast, accurate, and easy to implement.

[0076] In one embodiment, such as Figure 3 As shown, this involves determining the projection points of each edge point on a preset plane, and determining multiple projection vectors based on each edge point and its corresponding projection point. The steps of this implementation include:

[0077] Step 300: Determine the center point of the instrument panel based on the instrument image.

[0078] After acquiring the instrument image of the dashboard to be read, the controller can determine the center point of the dashboard by analyzing the image. The description of determining the center point of the dashboard based on the instrument image in this embodiment can be referenced to the specific description of determining the edge line of the dashboard in the instrument image in the above embodiment, and is not limited thereto. When the dashboard shape is circular, elliptical, or near-circular, the center point of the dashboard can be the center of the circle, ellipse, or near-circular shape; when the dashboard shape is rectangular or square, the center point of the dashboard can be the center point of the rectangle or square. This embodiment does not limit the specific location of the center point of the dashboard, as long as the function is achieved.

[0079] In an optional embodiment, after acquiring the instrument image, the controller can directly input the instrument image into the image analysis model, which can then output the edge line and center line of the instrument panel.

[0080] Step 310: Construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system.

[0081] After determining the center point of the dashboard, the controller can use this center point as the origin to construct a target coordinate system. The target coordinate system can be a two-dimensional or a three-dimensional coordinate system. The number axes in the target coordinate system can be the same as the number axes of the coordinate system in which the robotic arm is located. This embodiment does not impose restrictions on the constructed target coordinate system, as long as it can achieve its function.

[0082] After determining the target coordinate system, the controller marks the edge line of the instrument panel in the target coordinate system, determines multiple edge points on the edge line, and determines the first coordinates of each edge point on the edge line.

[0083] Step 320: Determine the second coordinates of the projection points of each edge point on the preset plane based on the first coordinates of each edge point.

[0084] The preset plane can be any plane in the target coordinate system. Specifically, the preset plane can be the plane formed by the X-axis and Y-axis in the target coordinate system.

[0085] After determining multiple edge points on the edge line in the target coordinate system, the controller determines the projection points of each edge point on the preset plane and determines the second coordinates of each projection point in the target coordinate system.

[0086] Step 330: Determine multiple projection vectors based on multiple first coordinates and corresponding second coordinates.

[0087] After determining multiple first coordinates of each edge point in the target coordinate system and the second coordinates of the corresponding projection points, the controller generates a projection vector for each edge point based on its first and second coordinates, thus obtaining multiple projection vectors corresponding to multiple edge points. The direction of the projection vector can point towards or away from the preset plane.

[0088] In this embodiment, the center point of the dashboard is determined based on the instrument image; a target coordinate system is constructed using the center point as the origin, and the first coordinates of each edge point in the target coordinate system are determined; based on the first coordinates of each edge point, the second coordinates of the projection points of each edge point on a preset plane are determined; and multiple projection vectors are determined based on multiple first coordinates and corresponding second coordinates. This method of determining projection vectors is quick and easy to implement.

[0089] In one embodiment, such as Figure 4 As shown, this involves an implementation method for determining multiple projection vectors based on multiple first coordinates and corresponding second coordinates. The steps of this implementation method include:

[0090] Step 400: For multiple first coordinates in each quadrant of the target coordinate system, determine multiple initial projection vectors based on the first coordinates and the corresponding second coordinates.

[0091] The target coordinate system comprises multiple quadrants, and the number of quadrants varies depending on the target coordinate system. For example, a two-dimensional coordinate system has four quadrants, while a three-dimensional coordinate system has eight quadrants. Specifically, if the target coordinate system is a three-dimensional coordinate system, all coordinate axes in the first quadrant are positive, and the coordinates in the first quadrant can be represented as (+X, +Y, +Z); in the second quadrant, the X-axis is negative, and the Y and Z axes are positive, and the coordinates in the second quadrant can be represented as (-X, +Y, +Z); in the third quadrant, the X and Y axes are negative, and the Z axis is positive, and the coordinates in the third quadrant can be represented as (-X, -Y, +Z); in the fourth quadrant, the X-axis is positive, the Y-axis is negative, and the Z-axis is negative, and the coordinates in the fourth quadrant can be represented as (+X, -Y, +Z). In the fifth quadrant, both the X and Y axes are positive, and the Z axis is negative. The coordinates of the fifth quadrant can be represented as (+X, +Y, -Z). In the sixth quadrant, the X axis is negative, the Y axis is positive, and the Z axis is negative. The coordinates of the sixth quadrant can be represented as (-X, +Y, -Z). In the seventh quadrant, all three axes are negative. The coordinates of the seventh quadrant can be represented as (-X, -Y, -Z). In the eighth quadrant, the X axis is positive, and both the Y and Z axes are negative. The coordinates of the eighth quadrant can be represented as (+X, -Y, -Z).

[0092] The number of edge points in each quadrant of the target coordinate system can be the same. For each edge point in each quadrant, multiple first coordinates can be used to form an initial projection vector based on the first coordinates of the edge point and the second coordinates of the projection point corresponding to that edge point. Thus, multiple initial projection vectors in that quadrant can be obtained.

[0093] In an optional embodiment, if the number of edge points in a certain quadrant is less than the number in other quadrants, or if there are no edge points in a certain quadrant, the initial projection vector corresponding to that quadrant can be supplemented according to a preset initial projection vector.

[0094] Step 410: Arrange multiple initial projection vectors in a preset order to obtain multiple projection vectors.

[0095] The preset order includes the first order within each quadrant and the second order between quadrants.

[0096] In an optional embodiment, the controller may number the edge points of each quadrant based on the edge lines, and the pre-order may refer to the order in which the edge points are numbered.

[0097] After obtaining multiple initial projection vectors in each quadrant of the target coordinate system, the controller arranges the multiple initial projection vectors in a pre-ordered manner and determines the multiple initial projection vectors after the arrangement as multiple projection vectors.

[0098] In this embodiment, for multiple first coordinates in each quadrant of the target coordinate system, multiple projection vectors are determined based on the first coordinates and their corresponding second coordinates. These initial projection vectors are then arranged in a preset order to obtain multiple projection vectors. This arrangement of the initial projection vectors in a preset order reflects the gradual distortion of the edge line. Furthermore, since the multiple projection vectors are arranged after being distinguished according to different quadrants, it facilitates the subsequent determination of the rotation angle based on the multiple projection vectors, thereby improving the accuracy of the determined rotation angle.

[0099] In one embodiment, such as Figure 5 As shown, the method further includes the following steps:

[0100] Step 500: For multiple initial projection vectors in each quadrant, determine the sub-adjustment matrix based on the multiple initial projection vectors.

[0101] After acquiring multiple initial projection vectors in each quadrant, the controller constructs sub-adjustment matrices based on these vectors, thus obtaining multiple sub-adjustment matrices corresponding to multiple quadrants. Each sub-adjustment matrix is ​​1×n or n×1 dimensional, where n represents the number of initial projection vectors in each quadrant.

[0102] Step 510: Perform ordered concatenation of multiple sub-adjustment matrices to determine the adjustment matrices corresponding to multiple projection vectors.

[0103] After determining the multiple sub-adjustment matrices corresponding to all quadrants, the controller can concatenate these sub-adjustment matrices in quadrant order to obtain adjustment matrices corresponding to multiple projection vectors. The adjustment matrices are 1×mn or mn×1 dimensional, where m represents the number of quadrants in the target coordinate system.

[0104] In this embodiment, for multiple initial projection vectors in each quadrant, a sub-adjustment matrix is ​​determined based on these initial projection vectors. These sub-adjustment matrices are then sequentially concatenated to determine the adjustment matrix corresponding to each projection vector. This converts the multiple projection vectors into matrix form, facilitating the subsequent determination of the rotation angle based on the adjustment matrix, thereby improving the accuracy of the determined rotation angle.

[0105] When the controller determines the adjustment angles corresponding to multiple projection vectors, one implementation method involves determining the rotation angle based on the multiple projection vectors. This implementation method includes:

[0106] Step 520: Determine the rotation angle based on the adjustment matrix corresponding to multiple projection vectors.

[0107] After determining the adjustment matrix corresponding to multiple projection vectors, the controller directly determines the rotation angle based on the adjustment matrix. A description of the rotation angle can be found in the detailed description of the above embodiments, and will not be repeated here.

[0108] In an optional embodiment, the controller can input the determined adjustment matrix into a pre-trained angle determination model, and the angle determination model outputs the rotation angle.

[0109] In this embodiment, the rotation angle is determined directly by the adjustment matrix corresponding to the determined multiple projection vectors, which can improve the accuracy of the determined rotation angle.

[0110] In one embodiment, such as Figure 6 As shown, one implementation method involves acquiring an instrument image of a dashboard to be read, and the steps of this implementation method include:

[0111] Step 600: Obtain the initial instrument image of the instrument panel and determine the distance between the instrument panel and the end effector of the robotic arm based on the initial instrument image.

[0112] The initial instrument image of the dashboard can be an image captured by a camera component located at the end of the robotic arm after the controller moves the robotic arm to a predetermined initial position; alternatively, it can be an instrument image selected from multiple images captured in real time by the camera component at the end of the robotic arm during the movement of the controller, containing all images of the dashboard. This embodiment does not limit the specific method for obtaining the initial instrument image of the dashboard, as long as the function can be achieved.

[0113] After acquiring the initial instrument image from the dashboard, the controller analyzes the image to determine the distance between the dashboard and the end effector of the robotic arm. This embodiment does not limit the specific method used to determine the distance between the dashboard and the end effector of the robotic arm, as long as the function is achieved.

[0114] In an optional embodiment, when the imaging component is a depth camera, the depth information of the area where the dashboard is located in the initial dashboard image is determined directly by analyzing the initial dashboard image, and the distance between the dashboard and the end effector of the robotic arm is determined based on the depth information.

[0115] Step 610: Determine the distance adjustment parameters based on the distance and distance threshold; control the movement of the robotic arm based on the distance adjustment parameters; and control the robotic arm to take pictures of the instrument panel using the imaging component after the movement to obtain the instrument image.

[0116] The distance threshold can be set by staff based on actual application scenarios and experience, and stored in the controller's storage unit.

[0117] After determining the distance between the instrument panel and the end effector of the robotic arm, the controller compares this distance with a preset distance threshold to determine the distance adjustment parameters.

[0118] In an optional embodiment, the distance threshold can be a first distance threshold and a second distance threshold, where the first distance threshold is greater than the second distance threshold, and the first and second distance thresholds can form a distance interval. The controller determines whether the distance is within this distance interval, that is, whether the distance is less than the first distance threshold and greater than the second distance threshold. If the distance is determined to be within this distance interval, it means that no adjustment is needed between the instrument panel and the end effector of the robotic arm, and the determined distance adjustment parameter can be zero. If the distance is determined to be outside the distance interval and greater than the first distance threshold, it means that the distance between the instrument panel and the end effector of the robotic arm is too far and needs to be shortened. The distance adjustment parameter can then be determined based on the difference between the distance and the first distance threshold. If the distance is determined to be outside the distance interval and less than the second distance threshold, it means that the distance between the instrument panel and the end effector of the robotic arm is too close and needs to be increased. The distance adjustment parameter can then be determined based on the difference between the distance and the second distance threshold.

[0119] After determining the distance adjustment parameters, the controller controls the robotic arm to move according to these parameters. Once the robotic arm has moved according to the distance adjustment parameters, the camera component at the end of the robotic arm captures an image of the instrument panel to be read, thus obtaining the instrument image.

[0120] In one specific embodiment, the determined distance adjustment parameter can carry a sign. If the distance adjustment parameter is positive, the robotic arm is controlled to move towards the direction where the instrument panel is located according to the absolute value of the distance adjustment parameter; if the distance adjustment parameter is negative, the robotic arm is controlled to move away from the instrument panel according to the absolute value of the distance adjustment parameter.

[0121] In this embodiment, an initial instrument image of the dashboard is first acquired, and the distance between the dashboard and the end effector of the robotic arm is determined based on the initial instrument image. Distance adjustment parameters are then determined based on the distance and a distance threshold. The robotic arm is then controlled to move according to the distance adjustment parameters, and after movement, the robotic arm uses a camera to capture an image of the dashboard. This way, after adjusting the distance between the end effector of the robotic arm and the dashboard for self-testing, the instrument image obtained using the camera is clearer, thereby improving the accuracy of the rotation angle subsequently determined based on the instrument image.

[0122] In one embodiment, such as Figure 7 As shown, this relates to an implementation method whereby a robotic arm, after moving, uses a camera component to capture an image of the instrument panel to obtain an instrument image. The steps of this implementation method include:

[0123] Step 700: Adjust the parameters according to the distance and adjust the focal length of the shooting component.

[0124] After determining the distance adjustment parameters, the controller adjusts the distance between the instrument panel and the end effector of the robotic arm according to these parameters. This determines the adjusted distance, and the focal length of the imaging component is then adjusted based on this adjusted distance. This embodiment does not limit the specific method for adjusting the focal length of the imaging component, as long as the function is achieved.

[0125] In an optional embodiment, the controller automatically adjusts the focal length of the lens of the imaging component based on an algorithm or mechanical adjustment mechanism.

[0126] Step 710: After the robotic arm moves, use the adjusted camera component to take a picture of the instrument panel to obtain an instrument image.

[0127] After the controller moves the robotic arm, it controls the focusing and shooting components to take pictures of the instrument panel, thus obtaining an image of the instrument panel.

[0128] In this embodiment, the focal length of the camera component is adjusted by adjusting the distance parameter. This ensures that when the camera component with the adjusted focal length is used to photograph the dashboard, the dashboard is clearly imaged on the imaging plane, thereby improving the accuracy and efficiency of the camera component.

[0129] In an optional embodiment, after determining the target instrument image, the robotic arm control method further includes:

[0130] The target distance between the instrument panel and the end effector of the robotic arm is determined based on the target instrument image, and the distance adjustment parameters are determined based on the target distance and the distance threshold; the movement of the robotic arm is controlled based on the distance adjustment parameters.

[0131] The description of determining the target distance between the instrument panel and the end effector of the robotic arm based on the target instrument image, determining the distance adjustment parameters based on the target distance and the distance threshold, and controlling the movement of the robotic arm based on the distance adjustment parameters can be found in the above embodiment, which describes the determination of the distance between the instrument panel and the end effector of the robotic arm based on the initial instrument image, the determination of the distance adjustment parameters based on the distance and the distance threshold, and the control of the movement of the robotic arm based on the distance adjustment parameters. It will not be repeated here.

[0132] After controlling the robotic arm to move, the camera component is controlled to take a picture of the instrument panel to obtain an image of the instrument panel. This image of the instrument panel can be identified as the instrument image of the instrument panel to be read, and then the process returns to execute steps 200-220 above.

[0133] In an optional embodiment, the robotic arm control system further includes a radar device positioned at the end effector of the robotic arm. In this case, the robotic arm control method further includes:

[0134] During the process of the controller controlling the rotation or movement of the robotic arm, the radar device collects the safe distance between the end of the robotic arm and the obstacle in real time; when the safe distance is equal to the preset distance, the controller stops the movement of the robotic arm.

[0135] Alternatively, during the process of the controller controlling the rotation of the robotic arm, the distance the end of the robotic arm moves towards the surrounding obstacles is calculated based on the angles of each rotation axis of the robotic arm; it is then determined whether this moving distance is less than the distance between the end of the robotic arm and the obstacle. If it is less, the robotic arm is controlled to stop moving while moving a preset safe distance.

[0136] Specifically, when there are multiple obstacles around the end of the robotic arm, the distance the end of the robotic arm moves toward each obstacle is calculated based on the angle of each rotation axis of the robotic arm; if the moving distance corresponding to any side is less than the distance between the end of the robotic arm and the obstacle, the robotic arm is controlled to stop moving while moving a preset safe distance.

[0137] In this embodiment, by using a radar device mounted on the robotic arm to collect the distance between the end of the robotic arm and surrounding obstacles in real time, collisions between the robotic arm and obstacles during movement or rotation can be avoided, thus improving safety. Furthermore, by setting a pre-defined safe distance, the maximum movement of the robotic arm can be ensured, thereby maximizing the accuracy of the acquired target instrument images.

[0138] Please see Figure 8 One embodiment of this application provides a robotic arm control method, the steps of which include:

[0139] Step 800: Obtain the initial instrument image of the instrument panel to be read, and determine the distance between the instrument panel and the end of the robotic arm based on the initial instrument image;

[0140] Step 810: Determine the distance adjustment parameters based on the distance and distance threshold; and adjust the focal length of the shooting component based on the distance adjustment parameters;

[0141] Step 820: Control the movement of the robotic arm according to the distance adjustment parameters, and after the robotic arm moves, use the shooting component with the adjusted focus to take a picture of the instrument panel to obtain the instrument image;

[0142] Step 830: Based on the instrument image, determine the center point and edge line of the instrument panel in the instrument image;

[0143] Step 840: Construct a three-dimensional coordinate system with the center point as the origin, mark the edge lines in the three-dimensional coordinate system, and determine multiple edge points on the edge lines;

[0144] Step 850: For a preset number of edge points in each quadrant of the three-dimensional coordinate system, determine the projection points of each edge point on a preset plane;

[0145] Step 860: Number each edge point according to the order from one end of the edge line to the other end, and determine multiple initial projection vectors according to the first coordinate of each edge point and the second coordinate of the corresponding projection point in the order of numbering.

[0146] Step 870: Construct sub-adjustment matrices based on multiple initial projection vectors, and concatenate the sub-adjustment matrices corresponding to all quadrants in an ordered manner to obtain the adjustment matrix;

[0147] Step 880: Input the adjustment matrix into the pre-trained angle determination model to determine the rotation angle; and control the rotation axis of the robotic arm to rotate according to the rotation angle, and control the robotic arm to acquire the target instrument image of the dashboard using the imaging component after rotation.

[0148] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0149] Based on the same inventive concept, this application also provides a robotic arm control device for implementing the robotic arm control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the robotic arm control device provided below can be found in the limitations of the robotic arm control method described above, and will not be repeated here.

[0150] In one embodiment, such as Figure 9 As shown, a robotic arm control device 10 is provided, including: an acquisition module 11, a determination module 12, and a control module 13, wherein:

[0151] The acquisition module 11 is used to acquire the instrument image of the instrument panel to be read, and to determine the edge line of the instrument panel in the instrument image based on the instrument image; the instrument image is captured by the imaging component set on the robotic arm;

[0152] The determination module 12 is used to determine the projection points of each edge point on the edge line on the preset plane, and to determine multiple projection vectors based on each edge point and its corresponding projection point.

[0153] The control module 13 is used to determine the rotation angle based on multiple projection vectors, control the rotation axis of the robotic arm to rotate based on the rotation angle, and control the robotic arm to take pictures of the instrument panel using the imaging component after rotation to obtain the target instrument image.

[0154] In one embodiment, the determining module 12 includes a first determining unit, a second determining unit, and a third determining unit. The first determining unit is used to determine the center point of the instrument panel based on the instrument image; the second determining unit is used to construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system; and determine the second coordinates of the projection points of each edge point on a preset plane based on the first coordinates of each edge point; the third determining unit is used to determine multiple projection vectors based on multiple first coordinates and corresponding second coordinates.

[0155] In one embodiment, the third determining unit is specifically used to determine multiple initial projection vectors based on multiple first coordinates in each quadrant of the target coordinate system and the corresponding second coordinates; and to arrange the multiple initial projection vectors in a preset order to obtain multiple projection vectors.

[0156] In one embodiment, the determining module 12 further includes a fourth determining unit and a stitching unit. The fourth determining unit is used to determine a sub-adjustment matrix based on the multiple initial projection vectors for each quadrant; the stitching unit is used to sequentially stitch the multiple sub-adjustment matrices to determine the adjustment matrix corresponding to the multiple projection vectors. The control module 13 is used to determine the rotation angle based on the adjustment matrix corresponding to the multiple projection vectors.

[0157] In one embodiment, the control module 13 is specifically used to input multiple projection vectors into a pre-trained angle determination model, and the angle determination model outputs a rotation angle.

[0158] In one embodiment, the acquisition module 11 includes an acquisition unit and a control unit. The acquisition unit is used to acquire an initial instrument image of the dashboard and determine the distance between the dashboard and the end effector of the robotic arm based on the initial instrument image. The control unit is used to determine a distance adjustment parameter based on the distance and a distance threshold; control the movement of the robotic arm based on the distance adjustment parameter; and control the robotic arm to take a picture of the dashboard using a camera component after movement to obtain an instrument image.

[0159] In one embodiment, the control unit is specifically used to adjust the focal length of the imaging component according to the distance adjustment parameter; after the robotic arm moves, it uses the adjusted imaging component to capture an image of the instrument panel to obtain an instrument image.

[0160] Each module in the aforementioned robotic arm control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0161] In one embodiment, a robotic arm control system is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0162] The instrument image of the instrument panel to be read is acquired, and the edge line of the instrument panel in the instrument image is determined based on the instrument image; the instrument image is obtained by capturing images from a camera component mounted on the robotic arm.

[0163] Determine the projection points of each edge point on the edge line onto a preset plane, and determine multiple projection vectors based on each edge point and its corresponding projection point;

[0164] The rotation angle is determined based on multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate according to the rotation angle. After the robotic arm rotates, it uses a camera component to capture an image of the instrument panel to obtain the target instrument image.

[0165] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the center point of the instrument panel based on the instrument image; constructing a target coordinate system with the center point as the origin, and determining the first coordinates of each edge point in the target coordinate system; determining the second coordinates of the projection points of each edge point on a preset plane based on the first coordinates of each edge point; and determining multiple projection vectors based on multiple first coordinates and corresponding second coordinates.

[0166] In one embodiment, when the processor executes the computer program, it further performs the following steps: for multiple first coordinates in each quadrant of the target coordinate system, determine multiple initial projection vectors based on the first coordinates and the corresponding second coordinates; arrange the multiple initial projection vectors in a preset order to obtain multiple projection vectors.

[0167] In one embodiment, when the processor executes the computer program, it further performs the following steps: for multiple initial projection vectors in each quadrant, determining a sub-adjustment matrix based on the multiple initial projection vectors; sequentially concatenating the multiple sub-adjustment matrices to determine the adjustment matrix corresponding to the multiple projection vectors; and determining the rotation angle based on the adjustment matrix corresponding to the multiple projection vectors.

[0168] In one embodiment, the processor, when executing the computer program, also performs the following steps: inputting multiple projection vectors into a pre-trained angle determination model, and the angle determination model outputting a rotation angle.

[0169] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring an initial instrument image of the dashboard and determining the distance between the dashboard and the end effector of the robotic arm based on the initial instrument image; determining a distance adjustment parameter based on the distance and a distance threshold; controlling the movement of the robotic arm based on the distance adjustment parameter; and controlling the robotic arm to take a picture of the dashboard using a camera component after the movement to obtain an instrument image.

[0170] In one embodiment, when the processor executes the computer program, it also performs the following steps: adjusting the focal length of the imaging component according to the distance adjustment parameters; controlling the robotic arm to take a picture of the instrument panel using the adjusted imaging component after moving, thereby obtaining an instrument image.

[0171] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0172] The instrument image of the instrument panel to be read is acquired, and the edge line of the instrument panel in the instrument image is determined based on the instrument image; the instrument image is obtained by capturing images from a camera component mounted on the robotic arm.

[0173] Determine the projection points of each edge point on the edge line onto a preset plane, and determine multiple projection vectors based on each edge point and its corresponding projection point;

[0174] The rotation angle is determined based on multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate according to the rotation angle. After the robotic arm rotates, it uses a camera component to capture an image of the instrument panel to obtain the target instrument image.

[0175] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the center point of the instrument panel based on the instrument image; constructing a target coordinate system with the center point as the origin, and determining the first coordinates of each edge point in the target coordinate system; determining the second coordinates of the projection points of each edge point on a preset plane based on the first coordinates of each edge point; and determining multiple projection vectors based on multiple first coordinates and corresponding second coordinates.

[0176] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for multiple first coordinates in each quadrant of the target coordinate system, determine multiple initial projection vectors based on the first coordinates and the corresponding second coordinates; arrange the multiple initial projection vectors in a preset order to obtain multiple projection vectors.

[0177] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for multiple initial projection vectors in each quadrant, determining a sub-adjustment matrix based on the multiple initial projection vectors; sequentially concatenating the multiple sub-adjustment matrices to determine the adjustment matrix corresponding to the multiple projection vectors; and determining the rotation angle based on the adjustment matrix corresponding to the multiple projection vectors.

[0178] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: inputting multiple projection vectors into a pre-trained angle determination model, and the angle determination model outputting a rotation angle.

[0179] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring an initial instrument image of the dashboard, and determining the distance between the dashboard and the end effector of the robotic arm based on the initial instrument image; determining a distance adjustment parameter based on the distance and a distance threshold; controlling the movement of the robotic arm based on the distance adjustment parameter; and controlling the robotic arm to take a picture of the dashboard using a camera component after the movement, thereby obtaining an instrument image.

[0180] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: adjusting the focal length of the imaging component according to the distance adjustment parameters; controlling the robotic arm to move and then using the adjusted imaging component to capture an image of the instrument panel to obtain an instrument image.

[0181] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0182] The instrument image of the instrument panel to be read is acquired, and the edge line of the instrument panel in the instrument image is determined based on the instrument image; the instrument image is obtained by capturing images from a camera component mounted on the robotic arm.

[0183] Determine the projection points of each edge point on the edge line onto a preset plane, and determine multiple projection vectors based on each edge point and its corresponding projection point;

[0184] The rotation angle is determined based on multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate according to the rotation angle. After the robotic arm rotates, it uses a camera component to capture an image of the instrument panel to obtain the target instrument image.

[0185] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the center point of the instrument panel based on the instrument image; constructing a target coordinate system with the center point as the origin, and determining the first coordinates of each edge point in the target coordinate system; determining the second coordinates of the projection points of each edge point on a preset plane based on the first coordinates of each edge point; and determining multiple projection vectors based on multiple first coordinates and corresponding second coordinates.

[0186] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for multiple first coordinates in each quadrant of the target coordinate system, determine multiple initial projection vectors based on the first coordinates and the corresponding second coordinates; arrange the multiple initial projection vectors in a preset order to obtain multiple projection vectors.

[0187] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for multiple initial projection vectors in each quadrant, determining a sub-adjustment matrix based on the multiple initial projection vectors; sequentially concatenating the multiple sub-adjustment matrices to determine the adjustment matrix corresponding to the multiple projection vectors; and determining the rotation angle based on the adjustment matrix corresponding to the multiple projection vectors.

[0188] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: inputting multiple projection vectors into a pre-trained angle determination model, and the angle determination model outputting a rotation angle.

[0189] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring an initial instrument image of the dashboard, and determining the distance between the dashboard and the end effector of the robotic arm based on the initial instrument image; determining a distance adjustment parameter based on the distance and a distance threshold; controlling the movement of the robotic arm based on the distance adjustment parameter; and controlling the robotic arm to take a picture of the dashboard using a camera component after the movement, thereby obtaining an instrument image.

[0190] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: adjusting the focal length of the imaging component according to the distance adjustment parameters; controlling the robotic arm to move and then using the adjusted imaging component to capture an image of the instrument panel to obtain an instrument image.

[0191] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0192] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0193] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robotic arm control method, characterized in that, The method includes: The instrument image of the instrument panel to be read is acquired, and the edge line of the instrument panel in the instrument image is determined based on the instrument image; the instrument image is obtained by capturing images using a camera component mounted on a robotic arm. Determine the projection points of each edge point on the edge line onto a preset plane, and determine multiple projection vectors based on each edge point and its corresponding projection point; The rotation angle is determined based on the multiple projection vectors, and the rotation axis of the robotic arm is controlled to rotate based on the rotation angle. After the robotic arm rotates, it is controlled to use the imaging component to capture an image of the instrument panel to obtain a target instrument image. The process of determining the projection points of each edge point on the edge line onto a preset plane, and determining multiple projection vectors based on each edge point and its corresponding projection point, includes: Based on the instrument image, determine the center point of the instrument panel; construct a target coordinate system using the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system; Based on the first coordinates of each edge point, determine the second coordinates of the projection points of each edge point on the preset plane; For each quadrant of the target coordinate system containing multiple first coordinates, multiple initial projection vectors are determined based on the first coordinates and the corresponding second coordinates. The multiple initial projection vectors are arranged in a preset order to obtain the multiple projection vectors.

2. The method according to claim 1, characterized in that, The method further includes: For each quadrant, a sub-adjustment matrix is ​​determined based on the plurality of initial projection vectors. The multiple sub-adjustment matrices are sequentially concatenated to determine the adjustment matrices corresponding to the multiple projection vectors; Determining the rotation angle based on the plurality of projection vectors includes: The rotation angle is determined based on the adjustment matrix corresponding to the plurality of projection vectors.

3. The method according to claim 1, characterized in that, Determining the rotation angle based on the plurality of projection vectors includes: The multiple projection vectors are input into a pre-trained angle determination model, which outputs the rotation angle.

4. The method according to any one of claims 1-3, characterized in that, The process of acquiring the instrument image of the instrument panel to be read includes: Acquire an initial instrument image of the instrument panel, and determine the distance between the instrument panel and the end effector of the robotic arm based on the initial instrument image; Based on the distance and distance threshold, a distance adjustment parameter is determined; and the movement of the robotic arm is controlled according to the distance adjustment parameter, and the robotic arm is controlled to take a picture of the instrument panel using the imaging component after the movement, so as to obtain the instrument image.

5. The method according to claim 4, characterized in that, The method of controlling the robotic arm to move and then using the imaging component to capture an image of the instrument panel to obtain an image of the instrument panel includes: The focal length of the shooting component is adjusted according to the distance adjustment parameters. After the robotic arm moves, it uses the adjusted imaging component to capture an image of the instrument panel, thereby obtaining the instrument image.

6. The method according to claim 4, characterized in that, The method further includes: During the process of controlling the rotation or movement of the robotic arm, the safe distance between the end of the robotic arm and the obstacle is collected by the radar at the end of the robotic arm. When the safe distance is equal to the preset distance, the robotic arm is controlled to stop moving.

7. A robotic arm control device, characterized in that, The apparatus, used in any one of claims 1-6, comprises: The acquisition module is used to acquire an instrument image of the instrument panel to be read, and to determine the edge line of the instrument panel in the instrument image based on the instrument image; the instrument image is acquired by a shooting component mounted on a robotic arm; The determining module is used to determine the projection points of each edge point on the edge line on a preset plane, and to determine multiple projection vectors based on each edge point and its corresponding projection point; The control module is used to determine the rotation angle based on the plurality of projection vectors, control the rotation axis of the robotic arm to rotate based on the rotation angle, and control the robotic arm to take a picture of the instrument panel using the imaging component after rotation to obtain a target instrument image.

8. A robotic arm control system, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images