Mechanical arm control method, device and system and readable storage medium

Abstract

The invention relates to a mechanical arm control method, device and system and a readable storage medium, and the method comprises the steps: obtaining an instrument image of a to-be-read instrument panel, and determining an edge line of the instrument panel in the instrument image according to the instrument image; the instrument image is obtained by shooting according to a shooting assembly arranged on the mechanical arm; determining a projection point of each edge point on the edge line on a preset plane, and determining a plurality of projection vectors according to each edge point and the corresponding projection point; and determining a rotation angle according to the plurality of projection vectors, controlling a rotation shaft of the mechanical arm to rotate according to the rotation angle, and controlling the mechanical arm to shoot the instrument panel by using a shooting assembly after rotation to obtain a target instrument image. According to the mechanical arm control method provided by the invention, a more accurate target instrument image can be obtained.

Description

Technical Field

[0001] This application relates to the field of control technologies, and particularly to a robotic arm control method, device, system, and readable storage medium. Background Art

[0002] In the grand scenario of smart power plants, robots are gradually becoming an indispensable and important role. With their characteristics of high efficiency, precision, and safety, they are profoundly changing the operation mode and energy efficiency management level of traditional power plants. In terms of the safety of power plants, robots are used to conduct inspections on power plants, and by taking images of instruments and collecting instrument data of each instrument, the automatic reading of instrument numbers is completed.

[0003] However, in traditional technologies, the images of instruments obtained by robots are inaccurate. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a robotic arm control method, device, system, and readable storage medium that can improve the accuracy of instrument images obtained by shooting.

[0005] In a first aspect, this application provides a robotic arm control method, which includes:

[0006] Obtain an instrument image of a dashboard to be read, and determine the edge line of the dashboard in the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm;

[0007] Determine the projection points of each edge point on the edge line on a preset plane, and determine a plurality of projection vectors according to each edge point and the corresponding projection point;

[0008] Determine the rotation angle according to the plurality of projection vectors, and control the rotation of the rotation axis of the robotic arm according to the rotation angle, and control the robotic arm to shoot the dashboard with the shooting component after rotation to obtain a target instrument image.

[0009] In one embodiment, determining the projection points of each edge point on the edge line on a preset plane, and determining a plurality of projection vectors according to each edge point and the corresponding projection point; includes:

[0010] Determine the center point of the dashboard according to the instrument image;

[0011] Construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system;

[0012] Determine the second coordinates of the projection points of each edge point on the preset plane according to the first coordinates of each edge point;

[0013] Determine a plurality of projection vectors according to a plurality of first coordinates and corresponding second coordinates.

[0014] In one embodiment, determining a plurality of projection vectors according to a plurality of first coordinates and corresponding second coordinates includes:

[0015] For a plurality of first coordinates in each quadrant in a target coordinate system, determine a plurality of initial projection vectors according to the first coordinates and corresponding second coordinates;

[0016] Arrange the plurality of initial projection vectors in a preset order to obtain a plurality of projection vectors.

[0017] In one embodiment, the method further includes:

[0018] For the plurality of initial projection vectors in each quadrant, determine a sub-adjustment matrix according to the plurality of initial projection vectors;

[0019] Perform an ordered splicing on the plurality of sub-adjustment matrices to determine an adjustment matrix corresponding to the plurality of projection vectors;

[0020] Determine a rotation angle according to the plurality of projection vectors, including:

[0021] Determine a rotation angle according to the adjustment matrix corresponding to the plurality of projection vectors.

[0022] In one embodiment, determining a rotation angle according to the plurality of projection vectors includes:

[0023] Input the plurality of projection vectors into a pre-trained angle determination model, and the angle determination model outputs a rotation angle.

[0024] In one embodiment, obtaining an instrument image of an instrument panel to be read includes:

[0025] Obtain an initial instrument image of the instrument panel, and determine the distance between the instrument panel and the end of the robotic arm according to the initial instrument image;

[0026] Determine a distance adjustment parameter according to the distance and a distance threshold; and control the robotic arm to move according to the distance adjustment parameter, and control the robotic arm to use a shooting component to shoot the instrument panel after moving to obtain an instrument image.

[0027] In one embodiment, controlling the robotic arm to use a shooting component to shoot the instrument panel after moving to obtain an instrument image includes:

[0028] Adjust the focal length of the shooting component according to the distance adjustment parameter;

[0029] Control the robotic arm to use the adjusted shooting component to shoot the instrument panel after moving to obtain an instrument image.

[0030] In a second aspect, an embodiment of the present application provides a robotic arm control device, which includes:

[0031] An acquisition module, configured to acquire an instrument image of the dashboard to be read, and determine the edge line of the dashboard in the instrument image according to the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm;

[0032] A determination module, configured to determine the projection points of each edge point on the edge line on a preset plane, and determine a plurality of projection vectors according to each edge point and the corresponding projection point;

[0033] A control module, configured to determine a rotation angle according to the plurality of projection vectors, control the rotation axis of the robotic arm to rotate according to the rotation angle, and control the robotic arm to shoot the dashboard with the shooting component after rotation to obtain a target instrument image.

[0034] In a third aspect, an embodiment of the present application further provides a robotic arm control system, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method provided in the first aspect are implemented.

[0035] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0036] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0037] The above robotic arm control method, device, system, and readable storage medium. The method includes obtaining an instrument image of the dashboard to be read, and determining the edge line of the dashboard in the instrument image according to the instrument image; the instrument image is obtained by a shooting component arranged on the robotic arm; determining the projection points of each edge point on the edge line on a preset plane, and determining a plurality of projection vectors according to each edge point and the corresponding projection point; determining a rotation angle according to the plurality of projection vectors, and controlling the rotation axis of the robotic arm to rotate according to the rotation angle, and controlling the robotic arm to use the shooting component to shoot the dashboard after rotation to obtain a target instrument image. In this embodiment, the plurality of projection vectors formed by each edge point on the edge line of the dashboard and the corresponding projection point can represent the distortion degree of the dashboard. By controlling the rotation axis of the robotic arm to rotate according to the rotation angle determined according to the plurality of projection vectors, the robotic arm can be rotated to the optimal angle, and the robotic arm is controlled to use the shooting component to shoot the dashboard after rotation, so that the obtained target instrument image can be prevented from being distorted, thereby improving the quality of the obtained target instrument image, and further improving the accuracy of the instrument data read based on the target instrument image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic structural diagram of a computer device in an embodiment;

[0039] Figure 2 It is a schematic flow chart of the steps of a robotic arm control method in an embodiment;

[0040] Figure 3 It is a schematic flow chart of the steps of a robotic arm control method in another embodiment;

[0041] Figure 4 It is a schematic flow chart of the steps of a robotic arm control method in another embodiment;

[0042] Figure 5 It is a schematic flow chart of the steps of a robotic arm control method in another embodiment;

[0043] Figure 6 It is a schematic flow chart of the steps of a robotic arm control method in another embodiment;

[0044] Figure 7 It is a schematic flow chart of the steps of a robotic arm control method in another embodiment;

[0045] Figure 8 It is a schematic flow chart of the steps of a robotic arm control method in another embodiment;

[0046] Figure 9 It is a schematic structural diagram of a robotic arm control device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In order to make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further elaborates on this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0048] The serial numbers assigned to components in this article, such as "first", "second", etc., are only used to distinguish the described objects and do not have any sequential or technical meanings.

[0049] First, before specifically introducing the technical solutions of the disclosed embodiments of this application, the background technology or the technical evolution context on which the embodiments of this application are based will be introduced. In the grand scenario of smart power plants, robots are gradually becoming an indispensable and important role. With their characteristics of high efficiency, precision, and safety, they are profoundly changing the operation mode and energy efficiency management level of traditional power plants. As the forefront of the digital transformation of the energy industry, smart power plants integrate advanced technologies such as the Internet of Things, big data, and artificial intelligence, and robots are vivid practices for the implementation and application of these technologies. The robots in smart power plants also have a high degree of environmental adaptability and learning ability, and can be customized according to the actual operation needs of the power plant, continuously optimizing the work process and improving the operation efficiency. Through seamless docking with the power plant management system, the data collected by the robots can be immediately uploaded to the cloud platform, providing data support for the intelligent decision-making of the power plant and helping to achieve more efficient, environmentally friendly, and sustainable energy production. In terms of daily inspections, relying on their excellent mobility and high-precision sensors, robots can independently perform detailed inspections of various areas of the power plant, including key parts such as generator sets, transmission lines, and cooling systems, effectively replacing manual labor to complete inspection tasks in high-risk or hard-to-reach areas. Robots can not only collect real-time equipment operation data but also, through built-in image recognition and thermal imaging analysis technologies, promptly detect potential fault points or overheating phenomena, providing strong support for preventive maintenance and significantly reducing equipment failure rates and unplanned downtime. In terms of the safety of the power plant, robots conduct inspections of the power plant and collect the instrument data of each instrument by taking pictures of the instruments to complete the automatic reading of the instrument numbers. However, in traditional technologies, when taking pictures of instruments by robots, it is difficult to ensure the best shooting angle, which may lead to inaccurate images of the instruments being taken, and further result in inaccurate instrument numbers obtained based on the images of the instruments. For this reason, this application provides a robotic arm control method.

[0050] The robotic arm control method provided by this application can be applied to a robotic arm control system. The robotic arm control system includes a controller, a moving component, a robotic arm installed on the moving component, and a shooting component arranged at the end of the robotic arm. A driving component for movement is arranged at the bottom of the moving component, and the driving component can be a moving foot, a moving wheel, or a moving track. The shooting component at the end of the robotic arm can be a depth camera. A depth camera is a camera that can capture the distance information between the photographed object and the depth camera and generate a three-dimensional scene. The depth camera utilizes depth sensor and image processor technologies and realizes the perception and measurement of the depth of the photographed object through different working principles, such as three-dimensional structured light technology, binocular vision technology, or Time of Flight (TOF) technology, etc. Among them, in the three-dimensional structured light technology, the depth camera emits light and receives the signal reflected back, and determines the distance between the object to be photographed and the depth camera by calculating the time difference or phase shift of the light.

[0051] The controller can be a computer device. As Figure 1 shown, the computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it realizes a robotic arm control method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0052] Those skilled in the art can understand that Figure 1 the structure shown in

[0053] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0054] In one embodiment, as Figure 2As shown, a robotic arm control method is provided. In this embodiment, the method is exemplified by being applied to a controller in a robotic arm control system. In this embodiment, the method includes the following steps:

[0055] Step 200: Obtain the instrument image of the dashboard to be read, and determine the edge line of the dashboard in the instrument image according to the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm.

[0056] The dashboard to be read can be the dashboards of various instruments in the power system. The instruments in the power system can include temperature instruments, pressure instruments, current instruments, flow instruments, electric instruments, etc. The shapes of various dashboards can be circular, oval, quasi-circular, square, fan-shaped, etc.

[0057] After the controller in the robotic arm control system controls the robotic arm to move to the initial position, it controls the shooting component at the end of the robotic arm to shoot the dashboard to be read and obtain the instrument image. The initial position can be preset by the user or the position reached by the robotic arm during the automatic movement according to the preset program.

[0058] After the controller obtains the instrument image, it can determine the edge line of the dashboard in the instrument image by analyzing the instrument image. The edge line of the dashboard refers to the line or contour around the dial or display screen of the dashboard. In this embodiment, there is no limitation on the specific method for determining the edge line of the dashboard in the instrument image, as long as its function can be realized.

[0059] In an optional embodiment, an image analysis model is pre-stored in the controller. The controller inputs the obtained instrument image into the image analysis model, and the edge line of the dashboard in the instrument image can be obtained. The controller can also perform feature analysis on the instrument image to determine the edge line of the dashboard.

[0060] Step 210: Determine the projection points of the respective edge points on the edge line on a preset plane, and determine a plurality of projection vectors according to the respective edge points and the corresponding projection points.

[0061] The preset plane can be any plane in the coordinate system where the robotic arm is located, or any plane in a coordinate system determined according to the dashboard image. Specifically, the preset plane can be a plane parallel to the ground where the robotic arm is located. In this embodiment, there is no limitation on the preset plane, as long as its function can be realized.

[0062] After the controller determines the edge line of the dashboard in the instrument image, it determines the respective edge points on the edge line. In this embodiment, there is no limitation on the specific method for determining the respective edge points on the edge line and the number of the respective edge points determined on the edge line, as long as its function can be realized.

[0063] In an optional embodiment, the controller may collect each point on the edge line at a preset interval and determine each collected point as an edge point. The preset intervals between the edge points may be the same or different.

[0064] After the controller determines each edge point on the edge line, for each edge point, it determines the projection point of the edge point on the preset plane. Based on the edge point and the corresponding projection point, the projection vector corresponding to the edge point can be determined, and thus multiple projection vectors corresponding to the edge points can be obtained. The direction of the projection vector may be the direction from the edge point to the corresponding projection point or the direction from the projection point to the corresponding edge point. The projection vector corresponding to each edge point can represent the distortion degree of the edge point.

[0065] In an optional embodiment, if the preset plane is parallel to the plane where the dashboard is located, indicating that the robotic arm is currently at the best shooting angle, the rotation angle determined subsequently based on the multiple projection vectors can be zero. If the preset plane is not parallel to the plane where the dashboard is located, the multiple determined projection vectors can represent the distortion degree of the instrument image, facilitating the subsequent determination of the rotation angle to control the rotation of the rotation axis of the robotic arm.

[0066] Step 220: Determine the rotation angle according to the multiple projection vectors, control the rotation axis of the robotic arm to rotate according to the rotation angle, and control the robotic arm to use the shooting component to shoot the dashboard after rotation to obtain the target instrument image.

[0067] After the controller determines the multiple projection vectors, it can determine the rotation angle according to the multiple projection vectors. The rotation angles determined according to the multiple projection vectors may include multiple ones. The number of rotation angles is the same as the number of rotation axes of the robotic arm. For example, if the robotic arm includes three rotation axes corresponding to rotations in three directions in three-dimensional space, the determined rotation angles include three angles, and each rotation axis corresponds to a rotation angle. If the robotic arm includes six rotation axes corresponding to six degrees of freedom directions in three-dimensional space, the six degrees of freedom directions include three translational degrees of freedom and three rotational degrees of freedom.

[0068] After the controller controls the rotation axis of the robotic arm to rotate according to the rotation angle, it controls the shooting component provided at the end of the robotic arm to shoot the dashboard to be read, and the target instrument image of the dashboard can be obtained.

[0069] In an optional embodiment, after the controller obtains the target instrument image, it can read the data in the dashboard by analyzing the target instrument image. Specifically, the controller may input the target instrument image into a pre-trained data reading model, and the data in the dashboard can be output through the data reading model.

[0070] The robotic arm control method provided by the embodiment of the present application obtains the instrument image of the dashboard to be read, and determines the edge line of the dashboard in the instrument image according to the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm; determines the projection points of each edge point on the edge line on a preset plane, and determines a plurality of projection vectors according to each edge point and the corresponding projection point; determines the rotation angle according to the plurality of projection vectors, and controls the rotation of the rotation axis of the robotic arm according to the rotation angle, and controls the robotic arm to shoot the dashboard with the shooting component after rotation to obtain a target instrument image. In this embodiment, a plurality of projection vectors formed by each edge point on the edge line of the dashboard and the corresponding projection point can represent the distortion degree of the dashboard. By controlling the rotation of the rotation axis of the robotic arm according to the rotation angle determined according to the plurality of projection vectors, the robotic arm can be rotated to the optimal angle, and the robotic arm is controlled to shoot the dashboard with the shooting component after rotation, so that the obtained target instrument image can be prevented from being distorted, thereby improving the quality of the obtained target instrument image, and further improving the accuracy of the instrument data read based on the target instrument image.

[0071] In one embodiment, an implementation manner of determining the rotation angle according to a plurality of projection vectors is involved. This implementation manner includes:

[0072] Input the plurality of projection vectors into a pre-trained angle determination model, and the angle determination model outputs the rotation angle.

[0073] The angle determination model can be obtained by training a neural network model with training samples and pre-stored in the storage unit of the controller. The training samples can include projection vector samples and standard rotation angles. The neural network model can be a convolutional neural network model or a multi-layer perceptron, etc. The type of the angle determination model is the same as that of the neural network model. The type and structure of the neural network model in this embodiment are not limited as long as its functions can be realized.

[0074] After the controller obtains the plurality of projection vectors, it inputs the plurality of projection vectors into the angle determination model. Through the analysis and processing of the angle determination model, the rotation angle can be output.

[0075] In this embodiment, by directly inputting the plurality of projection vectors into the pre-trained angle determination model, the rotation angle can be determined. The method of determining the rotation angle in this way is fast, accurate, and easy to implement.

[0076] In one embodiment, as Figure 3 shown, an implementation manner of determining the projection points of each edge point on the edge line on a preset plane and determining a plurality of projection vectors according to each edge point and the corresponding projection point is involved. The steps of this implementation manner include:

[0077] Step 300: Determine the center point of the instrument panel based on the instrument image.

[0078] After the controller obtains the instrument image of the instrument panel to be read, it can determine the center point of the instrument panel by analyzing the instrument image. The description of determining the center point of the instrument panel based on the instrument image in this embodiment can refer to the specific description of determining the edge line of the instrument panel in the instrument image in the above embodiment, which is not limited here. When the shape of the instrument panel is circular, oval or quasi-circular, the center point of the instrument panel can be the center of the circle, oval or quasi-circular; when the shape of the instrument panel is rectangular or square, the center point of the instrument panel can be the center point of the rectangle or square. The specific position of the center point of the instrument panel in this embodiment is not limited as long as its function can be realized.

[0079] In an alternative embodiment, after the controller obtains the instrument image, it can directly input the instrument image into the image analysis model, and the edge line and center line of the instrument panel can be output through the image analysis model.

[0080] Step 310: Construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system.

[0081] After the controller determines the center point of the instrument panel, it can construct a target coordinate system with this center point as the origin. The target coordinate system can be a two-dimensional coordinate system or a three-dimensional coordinate system. The number axes in the target coordinate system can be the same as the number axes in the coordinate system where the robotic arm is located. The constructed target coordinate system in this embodiment is not limited as long as its function can be realized.

[0082] After the controller determines the target coordinate system, it marks the edge line of the instrument panel in the target coordinate system, determines multiple edge points on the edge line, and determines the first coordinates of each edge point on this edge line.

[0083] Step 320: Determine the second coordinates of the projection points of each edge point on the preset plane according to the first coordinates of each edge point.

[0084] The preset plane can be any plane in the target coordinate system. Specifically, the preset plane can be the plane formed by the X-axis and Y-axis in the target coordinate system.

[0085] After the controller determines multiple edge points on the edge line in the target coordinate system, it determines the projection points of each edge point on the preset plane, and determines the second coordinates of each projection point in the target coordinate system.

[0086] Step 330: Determine multiple projection vectors according to the multiple first coordinates and the corresponding second coordinates.

[0087] After the controller determines the multiple first coordinates of each edge point in the target coordinate system and the second coordinates of the corresponding projection points, for each edge point, a projection vector is formed according to the first coordinate of the edge point and the corresponding second coordinate, so that multiple projection vectors corresponding to multiple edge points can be obtained. The direction of the projection vector can point to the direction of the preset plane or be opposite to the direction of the preset plane.

[0088] In this embodiment, the center point of the instrument panel is determined according to the instrument image; the center point is used as the origin to construct the target coordinate system, and the first coordinates of each edge point in the target coordinate system are determined; according to the first coordinates of each edge point, the second coordinates of the projection points of each edge point on the preset plane are determined; according to the multiple first coordinates and the corresponding second coordinates, multiple projection vectors are determined. The method of determining the projection vector in this way is fast and easy to implement.

[0089] In one embodiment, as Figure 4 shown, it relates to an implementation manner of determining multiple projection vectors according to multiple first coordinates and corresponding second coordinates. The steps of this implementation manner include:

[0090] Step 400: For the multiple first coordinates in each quadrant of the target coordinate system, determine multiple initial projection vectors according to the first coordinates and the corresponding second coordinates.

[0091] The target coordinate system includes multiple quadrants. Different target coordinate systems have different numbers of included quadrants. For example, a two-dimensional coordinate system includes 4 quadrants, and a three-dimensional coordinate system includes 8 quadrants. Specifically, if the target coordinate system is a three-dimensional coordinate system, all coordinate axes in the first quadrant are positive, and the coordinates of the first quadrant can be expressed as (+X, +Y, +Z); in the second quadrant, the X-axis is negative, and the Y-axis and Z-axis are positive, and the coordinates of the second quadrant can be expressed as (-X, +Y, +Z); in the third quadrant, both the X-axis and Y-axis are negative, and the Z-axis is positive, and the coordinates of the third quadrant can be expressed as (-X, -Y, +Z); in the fourth quadrant, the X-axis is positive, the Y-axis is negative, and the Z-axis is negative, and the coordinates of the fourth quadrant can be expressed as (+X, -Y, +Z); in the fifth quadrant, both the X-axis and Y-axis are positive, and the Z-axis is negative, and the coordinates of the fifth quadrant can be expressed as (+X, +Y, -Z); in the sixth quadrant, the X-axis is negative, the Y-axis is positive, and the Z-axis is negative, and the coordinates of the sixth quadrant can be expressed as (-X, +Y, -Z); in the seventh quadrant, the X-axis, Y-axis, and X-axis are all negative, and the coordinates of the seventh quadrant can be expressed as (-X, -Y, -Z); in the eighth quadrant, the X-axis is positive, and the Y-axis and Z-axis are both negative, and the coordinates of the eighth quadrant can be expressed as (+X, -Y, -Z).

[0092] The number of edge points in each quadrant of the target coordinate system can be the same. For the multiple first coordinates of each edge point in each quadrant, based on the first coordinate of the edge point and the second coordinate of the corresponding projection point, an initial projection vector can be formed, so that multiple initial projection vectors in this quadrant can be obtained.

[0093] In an optional embodiment, if the number of edge points in a certain quadrant is less than that in other quadrants, or there are no edge points in a certain quadrant, the initial projection vectors corresponding to this quadrant can be supplemented according to the preset initial projection vectors.

[0094] Step 410: Arrange the multiple initial projection vectors in a preset order to obtain multiple projection vectors.

[0095] The preset order includes the first order in each quadrant and the second order between quadrants.

[0096] In an optional embodiment, the controller can number the edge points of each quadrant based on the edge line, and the preset order can refer to the order of the numbers of the edge points.

[0097] After the controller obtains the multiple initial projection vectors in each quadrant of the target coordinate system, it arranges the multiple initial projection vectors in the preset order and determines the arranged multiple initial projection vectors as multiple projection vectors.

[0098] In this embodiment, for the multiple first coordinates in each quadrant of the target coordinate system, based on the first coordinates and the corresponding second coordinates, multiple projection vectors are determined; the multiple initial projection vectors are arranged in a preset order to obtain multiple projection vectors. Arranging the multiple initial projection vectors in a preset order like this can reflect the gradual deformation of the distortion degree of the edge line, and moreover, the multiple projection vectors are arranged after being distinguished by different quadrants, which is convenient for subsequently determining the rotation angle based on the multiple projection vectors and can improve the accuracy of the determined rotation angle.

[0099] In one embodiment, as Figure 5 shown, the steps of this method further include:

[0100] Step 500: For the multiple initial projection vectors of each quadrant, determine a sub-adjustment matrix according to the multiple initial projection vectors.

[0101] After the controller obtains the multiple initial projection vectors in each quadrant, it constructs a sub-adjustment matrix according to the multiple initial projection vectors, so that multiple sub-adjustment matrices corresponding to multiple quadrants can be obtained. Each sub-adjustment matrix is 1×n-dimensional or n×1-dimensional, where n represents the number of initial projection vectors in each quadrant.

[0102] Step 510: Arrange multiple sub-adjustment matrices in order and determine the adjustment matrix corresponding to multiple projection vectors.

[0103] After the controller determines multiple sub-adjustment matrices corresponding to all quadrants, it can splice the multiple sub-adjustment matrices in the order of the quadrants to obtain the adjustment matrix corresponding to multiple projection vectors. The adjustment matrix is 1×mn-dimensional or mn×1-dimensional, where m represents the number of quadrants in the target coordinate system.

[0104] In this embodiment, for multiple initial projection vectors in each quadrant, a sub-adjustment matrix is determined according to the multiple initial projection vectors; the multiple sub-adjustment matrices are arranged in order to determine the adjustment matrix corresponding to multiple projection vectors. In this way, multiple projection vectors are converted into matrix form, which is convenient for subsequently determining the rotation angle according to the adjustment matrix, thereby improving the accuracy of determining the rotation angle.

[0105] When the controller determines the adjustment angles corresponding to multiple projection vectors, an implementation method for determining the rotation angle according to multiple projection vectors is involved. This implementation method includes:

[0106] Step 520: Determine the rotation angle according to the adjustment matrix corresponding to multiple projection vectors.

[0107] After the controller determines the adjustment matrix corresponding to multiple projection vectors, it directly determines the rotation angle according to this adjustment matrix. The description of the rotation angle can refer to the specific description of the above embodiment and will not be elaborated here.

[0108] In an alternative embodiment, the controller can input the determined adjustment matrix into a pre-trained angle determination model, and the rotation angle is output through the angle determination model.

[0109] In this embodiment, the rotation angle is directly determined by the determined adjustment matrix corresponding to multiple projection vectors, which can improve the accuracy of the determined rotation angle.

[0110] In one embodiment, as Figure 6 shown, an implementation method for obtaining the instrument image of the dashboard to be read is involved. The steps of this implementation method include:

[0111] Step 600: Obtain the initial instrument image of the dashboard and determine the distance between the dashboard and the end of the robotic arm according to the initial instrument image.

[0112] The initial instrument image of the instrument panel can be the instrument image obtained by the shooting component arranged at the end of the robotic arm after the controller controls the robotic arm to move to a preset initial position; or it can be the instrument image of the entire image containing the instrument panel selected from multiple images taken in real time by the shooting component arranged at the end of the robotic arm during the process of controlling the robotic arm to move. This embodiment does not limit the specific method for obtaining the initial instrument image of the instrument panel, as long as its function can be realized.

[0113] After the controller obtains the initial instrument image of the instrument panel, it analyzes the initial instrument image to determine the distance between the instrument panel and the end of the robotic arm. This embodiment does not limit the specific method for determining the distance between the instrument panel and the end of the robotic arm, as long as its function can be realized.

[0114] In an optional embodiment, when the shooting component is a depth camera, the depth information of the area where the instrument panel is located in the initial instrument image is directly determined by analyzing the initial instrument image, and the distance between the instrument panel and the end of the robotic arm is determined according to this depth information.

[0115] Step 610: Determine the distance adjustment parameter according to the distance and the distance threshold; and control the movement of the robotic arm according to the distance adjustment parameter, and control the robotic arm to take a picture of the instrument panel with the shooting component after the movement to obtain an instrument image.

[0116] The distance threshold can be set by the staff according to the actual application scenario and experience and stored in the storage unit of the controller.

[0117] After the controller determines the distance between the instrument panel and the end of the robotic arm, it compares this distance with the preset distance threshold to determine the distance adjustment parameter.

[0118] In an alternative embodiment, the distance threshold may include a first distance threshold and a second distance threshold, where the first distance threshold is greater than the second distance threshold, and the first distance threshold and the second distance threshold may form a distance interval. The controller determines whether the distance is within this distance interval, that is, determines whether the distance is less than the first distance threshold and greater than the second distance threshold. If it is determined that the distance is within this distance range, it indicates that no adjustment is required for the distance between the dashboard and the end of the robotic arm, and the determined distance adjustment parameter may be zero. If it is determined that the distance is not within the distance interval and the distance is greater than the first distance threshold, it indicates that the distance between the dashboard and the end of the robotic arm is relatively far and needs to be shortened. Then, the distance adjustment parameter may be determined based on the difference between the distance and the first distance threshold. If it is determined that the distance is not within the distance interval and the distance is less than the second distance threshold, it indicates that the distance between the dial and the end of the robotic arm is relatively close and needs to be enlarged. Then, the distance adjustment parameter may be determined based on the difference between the distance and the second distance threshold.

[0119] After the controller determines the distance adjustment parameter, it controls the movement of the robotic arm according to this distance adjustment parameter, that is, controls the robotic arm to move according to the distance adjustment parameter. After the robotic arm moves according to the distance adjustment parameter, it controls the imaging component on the end of the robotic arm to capture the dashboard to be read, obtaining an instrument image.

[0120] In a specific embodiment, the determined distance adjustment parameter may carry a sign. If the distance adjustment parameter is positive, the robotic arm is controlled to move in the direction of the dashboard according to the absolute value of the distance adjustment parameter; if the distance adjustment parameter is negative, the robotic arm is controlled to move in the direction away from the dashboard according to the absolute value of the distance adjustment parameter.

[0121] In this embodiment, first, the initial instrument image of the dashboard is obtained, and the distance between the dashboard and the end of the robotic arm is determined according to the initial instrument image; according to the distance and the distance threshold, the distance adjustment parameter is determined; and according to the distance adjustment parameter, the movement of the robotic arm is controlled, and after the robotic arm moves, the imaging component is used to capture the dashboard to obtain an instrument image. In this way, after adjusting the distance between the end of the robotic arm and the dashboard self-check, the instrument image obtained by using the imaging component is clearer, thereby improving the accuracy of the rotation angle determined according to the instrument image subsequently.

[0122] In one embodiment, as Figure 7 shown, it relates to an implementation method of controlling the robotic arm to use the imaging component to capture the dashboard after moving to obtain an instrument image. The steps of this implementation method include:

[0123] Step 700: Adjust the focal length of the imaging component according to the distance adjustment parameter.

[0124] After determining the distance adjustment parameter, the controller adjusts the distance between the dashboard and the end of the robotic arm according to the distance adjustment parameter, and can determine the adjusted distance. Then, it adjusts the focal length of the shooting component according to the adjusted distance. This embodiment does not limit the specific method of adjusting the focal length of the shooting component, as long as its function can be realized.

[0125] In an alternative embodiment, the controller automatically adjusts the focal length of the lens of the shooting component through an algorithm or a mechanical adjustment mechanism.

[0126] Step 710: After controlling the robotic arm to move, use the adjusted shooting component to shoot the dashboard to obtain an instrument image.

[0127] After the controller controls the robotic arm to move, it controls the shooting component with the adjusted focal length to shoot the dashboard, and an instrument image can be obtained.

[0128] In this embodiment, the focal length of the photographing component is adjusted through the distance adjustment parameter, so as to ensure that when the dashboard is photographed using the shooting component with the adjusted focal length, the dashboard is clearly imaged on the imaging plane, thereby improving the accuracy and efficiency of the shooting by the shooting component.

[0129] In an alternative embodiment, after determining the target instrument image, the robotic arm control method further includes:

[0130] Determine the target distance between the dashboard and the end of the robotic arm according to the target instrument image, and determine the distance adjustment parameter according to the target distance and the distance threshold; control the robotic arm to move according to the distance adjustment parameter.

[0131] For the description of determining the target distance between the dashboard and the end of the robotic arm according to the target instrument image, determining the distance adjustment parameter according to the target distance and the distance threshold, and controlling the robotic arm to move according to the distance adjustment parameter, reference can be made to the specific description in the above embodiment of determining the distance between the dashboard and the end of the robotic arm according to the initial instrument image; determining the distance adjustment parameter according to the distance and the distance threshold; and controlling the robotic arm to move according to the distance adjustment parameter, which will not be elaborated here.

[0132] After controlling the robotic arm to move, control the shooting component to shoot the dashboard to obtain an image of the dashboard, and the image of the dashboard can be determined as the instrument image of the dashboard to be read, and return to execute the above steps 200 - 220.

[0133] In an alternative embodiment, the robotic arm control system further includes a radar device provided at the end of the robotic arm. In this case, the robotic arm control method further includes:

[0134] During the process of the controller controlling the rotation or movement of the robotic arm, the safety distance between the end of the robotic arm and the obstacle is collected in real time by the radar device; when the safety distance is equal to the preset distance, the controller controls the robotic arm to stop moving.

[0135] Alternatively, during the process of the controller controlling the rotation of the robotic arm, the moving distance of the end of the robotic arm moving towards the surrounding obstacles is calculated according to the angles of the respective rotating axes of the robotic arm; it is determined whether the moving distance is less than the distance between the end of the robotic arm and the obstacle, and if it is less, the controller controls the robotic arm to stop moving when the robotic arm moves a preset safety distance.

[0136] Specifically, when there are obstacles on multiple sides around the end of the robotic arm, the moving distances of the end of the robotic arm moving towards the obstacles on each side are calculated according to the angles of the respective rotating axes of the robotic arm; if there is any moving distance corresponding to a side face that is less than the distance between the end of the robotic arm and the obstacle, the controller controls the robotic arm to stop moving when the robotic arm moves a preset safety distance.

[0137] In this embodiment, by using the radar device provided on the robotic arm to collect the distance between the end of the robotic arm and the surrounding obstacles in real time, collisions between the robotic arm and the obstacles during the movement or rotation of the robotic arm can be avoided, the safety can be improved, and by setting the preset safety distance in advance, the maximum movement of the robotic arm can be ensured, so that the accuracy of the obtained target instrument image can be maximally improved.

[0138] Please refer to Figure 8 , an embodiment of the present application provides a method for controlling a robotic arm, and the steps of the method include:

[0139] Step 800: Obtain the initial instrument image of the dashboard to be read, and determine the distance between the dashboard and the end of the robotic arm according to the initial instrument image;

[0140] Step 810: Determine the distance adjustment parameter according to the distance and the distance threshold; and adjust the focal length of the shooting component according to the distance adjustment parameter;

[0141] Step 820: Control the movement of the robotic arm according to the distance adjustment parameter, and after the robotic arm moves, use the shooting component with the adjusted focal length to shoot the dashboard to obtain an instrument image;

[0142] Step 830: Determine the center point and the edge line of the dashboard in the instrument image according to the instrument image;

[0143] Step 840: Construct a three-dimensional coordinate system with the center point as the origin, mark the edge line in the three-dimensional coordinate system, and determine multiple edge points on the edge line;

[0144] Step 850: For a preset number of edge points in each quadrant of the three-dimensional coordinate system, determine the projection points of each edge point on the preset plane;

[0145] Step 860: Number the edge points in the order from one end to the other end of the edge line. According to the order of the numbers, determine multiple initial projection vectors based on the first coordinates of each edge point and the second coordinates of the corresponding projection points;

[0146] Step 870: Construct a sub-adjustment matrix based on the multiple initial projection vectors, and perform an ordered splicing on the sub-adjustment matrices corresponding to all quadrants to obtain an adjustment matrix;

[0147] Step 880: Input the adjustment matrix into a pre-trained angle determination model to determine the rotation angle; and control the rotation of the rotation axis of the robotic arm according to the rotation angle, and control the robotic arm to use the shooting component to obtain the target instrument image of the dashboard after rotation.

[0148] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0149] Based on the same inventive concept, an embodiment of the present application also provides a robotic arm control device for implementing the above-mentioned robotic arm control method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following robotic arm control devices can refer to the limitations on the robotic arm control method in the above text, and will not be repeated here.

[0150] In one embodiment, as Figure 9 shown, a robotic arm control device 10 is provided, including: an acquisition module 11, a determination module 12, and a control module 13, where:

[0151] The acquisition module 11 is configured to acquire the instrument image of the dashboard to be read, and determine the edge line of the dashboard in the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm;

[0152] A determination module 12, configured to determine the projection points of the edge points on the edge line on a preset plane, and determine a plurality of projection vectors according to the edge points and the corresponding projection points;

[0153] A control module 13, configured to determine a rotation angle according to the plurality of projection vectors, control the rotation of the rotation axis of the robotic arm according to the rotation angle, and control the robotic arm to use the shooting component to shoot the dashboard after rotation to obtain a target instrument image.

[0154] In one embodiment, the determination module 12 includes a first determination unit, a second determination unit, and a third determination unit. The first determination unit is configured to determine the center point of the dashboard according to the instrument image; the second determination unit is configured to construct a target coordinate system with the center point as the origin, and determine the first coordinates of the edge points in the target coordinate system; according to the first coordinates of the edge points, determine the second coordinates of the projection points of the edge points on the preset plane; the third determination unit is configured to determine a plurality of projection vectors according to the plurality of first coordinates and the corresponding second coordinates.

[0155] In one embodiment, the third determination unit is specifically configured to, for the plurality of first coordinates in each quadrant in the target coordinate system, determine a plurality of initial projection vectors according to the first coordinates and the corresponding second coordinates; arrange the plurality of initial projection vectors in a preset order to obtain a plurality of projection vectors.

[0156] In one embodiment, the determination module 12 further includes a fourth determination unit and a splicing unit. The fourth determination unit is configured to, for the plurality of initial projection vectors in each quadrant, determine a sub-adjustment matrix according to the plurality of initial projection vectors; the splicing unit is configured to perform an orderly splicing on the plurality of sub-adjustment matrices to determine an adjustment matrix corresponding to the plurality of projection vectors. The control module 13 is configured to determine the rotation angle according to the adjustment matrix corresponding to the plurality of projection vectors.

[0157] In one embodiment, the control module 13 is specifically configured to input the plurality of projection vectors into a pre-trained angle determination model, and the angle determination model outputs the rotation angle.

[0158] In one embodiment, the acquisition module 11 includes an acquisition unit and a control unit. The acquisition unit is configured to acquire an initial instrument image of the dashboard and determine the distance between the dashboard and the end of the robotic arm according to the initial instrument image. The control unit is configured to determine a distance adjustment parameter according to the distance and a distance threshold; and control the movement of the robotic arm according to the distance adjustment parameter, and control the robotic arm to use the shooting component to shoot the dashboard after movement to obtain an instrument image.

[0159] In one embodiment, the control unit is specifically configured to adjust the focal length of the shooting component according to the distance adjustment parameter; control the robotic arm to use the adjusted shooting component to shoot the dashboard after movement to obtain an instrument image.

[0160] Each module in the above robotic arm control device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0161] In one embodiment, a robotic arm control system is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0162] Obtain the instrument image of the dashboard to be read, and determine the edge line of the dashboard in the instrument image according to the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm;

[0163] Determine the projection points of each edge point on the edge line on a preset plane, and determine a plurality of projection vectors according to each edge point and the corresponding projection point;

[0164] Determine the rotation angle according to the plurality of projection vectors, and control the rotation axis of the robotic arm to rotate according to the rotation angle, and control the robotic arm to shoot the dashboard with the shooting component after rotation to obtain a target instrument image.

[0165] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determine the center point of the dashboard according to the instrument image; construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system; according to the first coordinates of each edge point, determine the second coordinates of the projection points of each edge point on the preset plane; determine a plurality of projection vectors according to the plurality of first coordinates and the corresponding second coordinates.

[0166] In one embodiment, when the processor executes the computer program, the following steps are also implemented: for the plurality of first coordinates in each quadrant of the target coordinate system, determine a plurality of initial projection vectors according to the first coordinates and the corresponding second coordinates; arrange the plurality of initial projection vectors in a preset order to obtain a plurality of projection vectors.

[0167] In one embodiment, when the processor executes the computer program, the following steps are also implemented: for the plurality of initial projection vectors in each quadrant, determine a sub-adjustment matrix according to the plurality of initial projection vectors; perform an orderly splicing on the plurality of sub-adjustment matrices to determine an adjustment matrix corresponding to the plurality of projection vectors; determine the rotation angle according to the adjustment matrix corresponding to the plurality of projection vectors.

[0168] In one embodiment, when the processor executes the computer program, the following steps are also implemented: input the plurality of projection vectors into a pre-trained angle determination model, and the angle determination model outputs the rotation angle.

[0169] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining an initial instrument image of the instrument panel, and determining the distance between the instrument panel and the end of the robotic arm according to the initial instrument image; determining a distance adjustment parameter according to the distance and a distance threshold; and controlling the movement of the robotic arm according to the distance adjustment parameter, and controlling the robotic arm to use the photographing component to photograph the instrument panel after the movement to obtain an instrument image.

[0170] In one embodiment, when the processor executes the computer program, the following steps are further implemented: adjusting the focal length of the photographing component according to the distance adjustment parameter; controlling the robotic arm to use the adjusted photographing component to photograph the instrument panel after the movement to obtain an instrument image.

[0171] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0172] Obtaining an instrument image of the instrument panel to be read, and determining the edge line of the instrument panel in the instrument image according to the instrument image; the instrument image is obtained by photographing using a photographing component arranged on the robotic arm;

[0173] Determining the projection points of the respective edge points on the edge line on a preset plane, and determining a plurality of projection vectors according to the respective edge points and the corresponding projection points;

[0174] Determining a rotation angle according to the plurality of projection vectors, and controlling the rotation axis of the robotic arm to rotate according to the rotation angle, and controlling the robotic arm to use the photographing component to photograph the instrument panel after the rotation to obtain a target instrument image.

[0175] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determining the center point of the instrument panel according to the instrument image; constructing a target coordinate system with the center point as the origin, and determining the first coordinates of the respective edge points in the target coordinate system; determining the second coordinates of the projection points of the respective edge points on the preset plane according to the first coordinates of the respective edge points; and determining a plurality of projection vectors according to the plurality of first coordinates and the corresponding second coordinates.

[0176] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: for the plurality of first coordinates in each quadrant in the target coordinate system, determining a plurality of initial projection vectors according to the first coordinates and the corresponding second coordinates; and arranging the plurality of initial projection vectors in a preset order to obtain a plurality of projection vectors.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: for a plurality of initial projection vectors in each quadrant, determine a sub-adjustment matrix according to the plurality of initial projection vectors; perform an ordered splicing on the plurality of sub-adjustment matrices to determine an adjustment matrix corresponding to the plurality of projection vectors; determine a rotation angle according to the adjustment matrix corresponding to the plurality of projection vectors.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: input the plurality of projection vectors into a pre-trained angle determination model, and the angle determination model outputs a rotation angle.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtain an initial instrument image of the dashboard, and determine the distance between the dashboard and the end of the robotic arm according to the initial instrument image; determine a distance adjustment parameter according to the distance and a distance threshold; and control the movement of the robotic arm according to the distance adjustment parameter, and control the robotic arm to use the shooting component to shoot the dashboard after the movement to obtain an instrument image.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: adjust the focal length of the shooting component according to the distance adjustment parameter; control the robotic arm to use the adjusted shooting component to shoot the dashboard after the movement to obtain an instrument image.

[0181] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0182] Obtain an instrument image of the dashboard to be read, and determine the edge line of the dashboard in the instrument image; the instrument image is obtained by shooting with a shooting component arranged on the robotic arm;

[0183] Determine the projection points of each edge point on the edge line on a preset plane, and determine a plurality of projection vectors according to each edge point and the corresponding projection point;

[0184] Determine a rotation angle according to the plurality of projection vectors, control the rotation of the rotation axis of the robotic arm according to the rotation angle, and control the robotic arm to use the shooting component to shoot the dashboard after the rotation to obtain a target instrument image.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: determine the center point of the dashboard according to the instrument image; construct a target coordinate system with the center point as the origin, and determine the first coordinates of each edge point in the target coordinate system; determine the second coordinates of the projection points of each edge point on the preset plane according to the first coordinates of each edge point; determine a plurality of projection vectors according to the plurality of first coordinates and the corresponding second coordinates.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: for a plurality of first coordinates in each quadrant of a target coordinate system, determine a plurality of initial projection vectors according to the first coordinates and corresponding second coordinates; arrange the plurality of initial projection vectors in a preset order to obtain a plurality of projection vectors.

[0187] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: for a plurality of initial projection vectors in each quadrant, determine a sub-adjustment matrix according to the plurality of initial projection vectors; perform an ordered splicing on the plurality of sub-adjustment matrices to determine an adjustment matrix corresponding to the plurality of projection vectors; determine a rotation angle according to the adjustment matrix corresponding to the plurality of projection vectors.

[0188] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: input the plurality of projection vectors into a pre-trained angle determination model, and the angle determination model outputs a rotation angle.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtain an initial instrument image of the dashboard, and determine the distance between the dashboard and the end of the robotic arm according to the initial instrument image; determine a distance adjustment parameter according to the distance and a distance threshold; and control the movement of the robotic arm according to the distance adjustment parameter, and control the robotic arm to capture the dashboard by using a capturing component after the movement to obtain an instrument image.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: adjust the focal length of the capturing component according to the distance adjustment parameter; control the robotic arm to capture the dashboard by using the adjusted capturing component after the movement to obtain an instrument image.

[0191] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0192] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0193] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A robot arm control method, characterized in that: The method comprises: Acquire an instrument image of an instrument panel to be read, and determine an edge line of the instrument panel in the instrument image according to the instrument image; the instrument image is obtained by photographing a photographing component arranged on a mechanical arm; Determine the projection point of each edge point on the edge line on a preset plane, and determine a plurality of projection vectors according to each edge point and the corresponding projection point; A rotation angle is determined according to the multiple projection vectors, and the rotation axis of the mechanical arm is controlled to rotate according to the rotation angle, and the mechanical arm is controlled to use the shooting component to shoot the instrument panel after the rotation to obtain a target instrument image.

2. The method according to claim 1, characterized in that The step of determining the projection points of each edge point on the edge line on a preset plane, and determining a plurality of projection vectors according to each edge point and the corresponding projection point, comprises: Determine the center point of the instrument panel according to the instrument image; Taking the center point as the origin to construct a target coordinate system, and determining the first coordinates of each edge point in the target coordinate system; Determining, according to the first coordinates of each edge point, the second coordinates of the projection point of each edge point on the preset plane; The plurality of projection vectors are determined according to the plurality of first coordinates and the corresponding second coordinates.

3. The method according to claim 2, characterized in that The determining the plurality of projection vectors according to the plurality of first coordinates and the corresponding second coordinates comprises: For a plurality of the first coordinates in each quadrant of the target coordinate system, determine a plurality of initial projection vectors according to the first coordinates and corresponding second coordinates; The multiple initial projection vectors are arranged in a preset order to obtain the multiple projection vectors.

4. The method according to claim 3, characterized in that The method further comprises: For the multiple initial projection vectors of each quadrant, determine a sub-adjustment matrix according to the multiple initial projection vectors; Orderly splicing the multiple sub-adjustment matrices to determine the adjustment matrices corresponding to the multiple projection vectors; The determining the rotation angle according to the multiple projection vectors comprises: The rotation angle is determined according to an adjustment matrix corresponding to the multiple projection vectors.

5. The method according to any one of claims 1 to 4, characterized in that The determining the rotation angle according to the multiple projection vectors comprises: The plurality of projection vectors are input into a pre-trained angle determination model, and the angle determination model outputs the rotation angle.

6. The method according to any one of claims 1 to 4, characterized in that The step of obtaining the instrument image of the instrument panel to be read includes: Acquire an initial instrument image of the instrument panel, and determine the distance between the instrument panel and the end of the mechanical arm according to the initial instrument image; According to the distance and the distance threshold, a distance adjustment parameter is determined; and according to the distance adjustment parameter, the movement of the robotic arm is controlled, and after the movement, the robotic arm is controlled to use the shooting component to shoot the instrument panel to obtain the instrument image.

7. The method according to claim 6, characterized in that The controlling the mechanical arm to photograph the instrument panel using the photographing component after the movement to obtain the instrument image includes: Adjusting the focal length of the photographing component according to the distance adjustment parameter; After the mechanical arm is controlled to move, the adjusted shooting component is used to shoot the instrument panel to obtain the instrument image.

8. A robot arm control device, characterized in that: The device comprises: An acquisition module, used for acquiring an instrument image of an instrument panel to be read, and determining an edge line of the instrument panel in the instrument image according to the instrument image; the instrument image is obtained by photographing a photographing component arranged on a mechanical arm; A determination module, used to determine the projection point of each edge point on the edge line on a preset plane, and determine a plurality of projection vectors according to each edge point and the corresponding projection point; The control module is used to determine the rotation angle according to the multiple projection vectors, and control the rotation axis of the mechanical arm according to the rotation angle, and control the mechanical arm to use the shooting component to shoot the instrument panel after the rotation to obtain a target instrument image.

9. A robot arm control system, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images