Musical instrument processing method and device based on image processing, upper computer and workstation

Abstract

The embodiment of the invention provides a musical instrument processing method and device based on image processing, an upper computer and a workstation. The method comprises the steps of obtaining a sound beam image of a to-be-installed sound beam corresponding to a target musical instrument; carrying out image processing on the sound beam image to obtain pose information of the sound beam to be installed; wherein the position and posture information represents the current position and posture of the sound beam to be installed; according to the pose information, the robot is controlled to conduct gluing operation treatment on the sound beam to be installed; and controlling the robot to install the sound beam to be installed on the target musical instrument. The method can ensure that the glue is uniformly distributed at the two ends of the sound beam, improve the bonding strength between the sound beam and the target musical instrument, and further improve the product quality of the dulcimer.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a musical instrument processing method, device, host computer and workstation based on image processing. Background Art

[0002] During the production process of musical instruments such as the dulcimer, the dulcimer needs to be processed, for example, the sound beam of the dulcimer needs to be installed.

[0003] In the prior art, a semi-automatic device is used to apply glue to both ends of the sound beam, and then the sound beam is installed in the soundboard of the dulcimer.

[0004] However, in the above method, the semi-automatic device cannot fully ensure the uniform distribution of glue at both ends of the sound beam, resulting in insufficient bonding strength, which in turn affects the product quality of the dulcimer. Summary of the invention

[0005] The embodiments of the present application provide a musical instrument processing method, device, host computer and workstation based on image processing, so as to improve the product quality of dulcimer.

[0006] In a first aspect, an embodiment of the present application provides a musical instrument processing method based on image processing, the method being applied to a host computer in a musical instrument processing workstation; the musical instrument processing workstation also includes a robot including:

[0007] Acquire a sound beam image of a sound beam to be installed corresponding to a target musical instrument; and perform image processing on the sound beam image to obtain position and posture information of the sound beam to be installed; wherein the position and posture information represents the current position and posture of the sound beam to be installed;

[0008] According to the posture information, controlling the robot to perform a gluing operation on the sound beam to be installed;

[0009] The robot is controlled to install the sound beam to be installed on the target musical instrument.

[0010] In a possible implementation, a flexible gripping tool is provided at the end of the robot; the musical instrument processing workstation further includes a torque sensor; and controlling the robot to install the sound beam to be installed on the target musical instrument includes:

[0011] According to the position information, controlling the flexible grasping tool of the robot to grasp the sound beam to be installed, so as to make the sound beam to be installed contact the soundboard of the target musical instrument;

[0012] Acquire the contact force information of the sound beam to be installed collected in real time by the torque sensor; wherein the contact force information represents the magnitude and direction of the contact force of the sound beam to be installed contacting the soundboard of the target musical instrument;

[0013] According to the contact force information, the robot is controlled to install the sound beam to be installed on the target musical instrument.

[0014] In a possible implementation, controlling the robot to install the sound beam to be installed on the target musical instrument according to the contact force information includes:

[0015] According to the first contact information and the second contact information in the contact force information, the robot is controlled to install the sound beam to be installed on the target musical instrument; wherein the first contact information represents the magnitude of the contact force in a first preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument; and the second contact information represents the magnitude of the contact force in a second preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument.

[0016] In a possible implementation, controlling the robot to install the sound beam to be installed on the target musical instrument according to the first contact information and the second contact information in the contact force information includes:

[0017] If it is determined that both the first contact information and the second contact information satisfy a preset condition, the robot is controlled to press down and fix the sound beam to be installed on the target musical instrument.

[0018] In a possible implementation, performing image processing on the sound beam image to obtain the position information of the sound beam to be installed includes:

[0019] Performing point cloud conversion processing on the sound beam image to obtain sound beam point cloud data corresponding to the sound beam image;

[0020] Template matching is performed on the sound beam point cloud data to obtain the position information of the sound beam to be installed.

[0021] In a possible implementation, template matching is performed on the sound beam point cloud data to obtain the position information of the sound beam to be installed, including:

[0022] Performing filtering and alignment processing on the sound beam point cloud data to obtain processed sound beam point cloud data;

[0023] Determine the template point cloud data corresponding to the sound beam point cloud data; and determine the matching parameters;

[0024] According to the matching parameters, the template point cloud data and the processed sound beam point cloud data are matched to obtain a matching result; wherein the matching result includes the position information of the sound beam to be installed.

[0025] In a possible implementation, a flexible gripping tool is provided at the end of the robot; and according to the posture information, the robot is controlled to perform a gluing operation on the sound beam to be installed, including:

[0026] According to the posture information, the flexible grasping tool of the robot is controlled to grasp the sound beam to be installed, so as to perform a gluing operation on the sound beam to be installed.

[0027] In a second aspect, an embodiment of the present application provides a musical instrument processing device based on image processing, the device is applied to a host computer in a musical instrument processing workstation; the musical instrument processing workstation also includes a robot; the device includes:

[0028] A processing module, used to obtain a sound beam image of a sound beam to be installed corresponding to a target musical instrument; and perform image processing on the sound beam image to obtain position and posture information of the sound beam to be installed; wherein the position and posture information represents a current position and posture of the sound beam to be installed;

[0029] A gluing module, used for controlling the robot to perform a gluing operation on the sound beam to be installed according to the posture information;

[0030] The installation module is used to control the robot to install the sound beam to be installed on the target musical instrument.

[0031] In a possible implementation, a flexible gripping tool is provided at the end of the robot; the musical instrument processing workstation also includes a torque sensor; the installation module is specifically used to: control the flexible gripping tool of the robot to grip the sound beam to be installed according to the posture information, so as to make the sound beam to be installed contact the sound board of the target musical instrument; obtain the contact force information of the sound beam to be installed collected in real time by the torque sensor; wherein the contact force information represents the size and direction of the contact force of the sound beam to be installed contacting the sound board of the target musical instrument; and control the robot to install the sound beam to be installed on the target musical instrument according to the contact force information.

[0032] In a possible implementation, the installation module is specifically used to: control the robot to install the sound beam to be installed on the target musical instrument according to the first contact information and the second contact information in the contact force information; wherein the first contact information represents the magnitude of the contact force in a first preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument; and the second contact information represents the magnitude of the contact force in a second preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument.

[0033] In a possible implementation, the installation module is specifically configured to: if it is determined that both the first contact information and the second contact information satisfy a preset condition, control the robot to press down and fix the sound beam to be installed on the target musical instrument.

[0034] In a possible implementation, the processing module is specifically used to: perform point cloud conversion processing on the sound beam image to obtain sound beam point cloud data corresponding to the sound beam image; perform template matching processing on the sound beam point cloud data to obtain posture information of the sound beam to be installed.

[0035] In a possible implementation, the processing module is specifically used to: perform filtering and alignment processing on the sound beam point cloud data to obtain processed sound beam point cloud data; determine the template point cloud data corresponding to the sound beam point cloud data; and determine matching parameters; according to the matching parameters, perform matching processing on the template point cloud data and the processed sound beam point cloud data to obtain a matching result; wherein the matching result includes the posture information of the sound beam to be installed.

[0036] In a possible implementation, a flexible gripping tool is provided at the end of the robot; the gluing module is specifically used to: control the flexible gripping tool of the robot to grip the sound beam to be installed according to the posture information, so as to perform a gluing operation on the sound beam to be installed.

[0037] In a third aspect, an embodiment of the present application provides a host computer, including: a memory, a processor;

[0038] The memory stores computer-executable instructions;

[0039] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a musical instrument processing workstation, comprising: a host computer and a robot, wherein the host computer is used to execute the above first aspect and / or various possible implementations of the first aspect.

[0041] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0042] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0043] The musical instrument processing method, device, host computer and workstation based on image processing provided in the embodiments of the present application perform image processing on the sound beam image of the sound beam to be installed corresponding to the target musical instrument, so as to obtain the current position and posture of the sound beam to be installed, and thereby control the robot to perform glue coating operation on the sound beam to be installed, and after the gluing is completed, control the robot to install the sound beam to be installed on the target musical instrument; thereby, it is possible to ensure the uniform distribution of glue at both ends of the sound beam, improve the bonding strength between the sound beam and the target musical instrument, and thereby improve the product quality of the dulcimer. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0045] Figure 1 A schematic diagram of a process flow of a musical instrument processing method based on image processing provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of the structure of a musical instrument processing workstation provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a musical instrument processing process provided in an embodiment of the present application;

[0048] Figure 4 A schematic flow chart of another musical instrument processing method based on image processing provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of a point cloud conversion process provided in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of an image processing process provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of a point cloud filtering process provided in an embodiment of the present application;

[0052] Figure 8 A schematic diagram of a template matching process provided in an embodiment of the present application;

[0053] Fig. 9 A schematic diagram of the structure of another musical instrument processing workstation provided in an embodiment of the present application;

[0054] Fig.10 A structural schematic diagram of another musical instrument processing workstation provided in an embodiment of the present application;

[0055] Fig.11A schematic diagram of the structure of a musical instrument processing device based on image processing provided in an embodiment of the present application;

[0056] Fig.12 A schematic diagram of the structure of a host computer provided in an embodiment of the present application.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0060] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0061] It should be noted that the present application can be used in the field of image processing, and can also be used in any field other than the field of image processing, and the application field of the present application is not limited.

[0062] The specific application scenario of the present application is: gluing the two ends of the sound beam is performed through a semi-automatic device, and then the sound beam is installed into the soundboard of the dulcimer.

[0063] Combined with the above scenario, it can be seen that the semi-automatic device cannot fully ensure the uniform distribution of glue on both ends of the sound beam, resulting in insufficient bonding strength, which in turn affects the product quality of the dulcimer.

[0064] The musical instrument processing method based on image processing provided in the present application obtains the current position and posture of the sound beam to be installed by performing image processing on the sound beam image of the sound beam to be installed corresponding to the target musical instrument, and controls the robot to perform glue application operation on the sound beam to be installed. After the gluing is completed, the robot is controlled to install the sound beam to be installed on the target musical instrument, thereby solving the technical problem that the semi-automatic device cannot fully ensure the uniform distribution of glue at both ends of the sound beam, thereby affecting the product quality of the dulcimer.

[0065] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] Figure 1 A flowchart of a musical instrument processing method based on image processing provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method is applied to a host computer in a musical instrument processing workstation; the musical instrument processing workstation also includes a robot; the method includes:

[0067] 201. Acquire a sound beam image of a sound beam to be installed corresponding to a target musical instrument; and perform image processing on the sound beam image to obtain position and posture information of the sound beam to be installed; wherein the position and posture information represents a current position and posture of the sound beam to be installed.

[0068] Exemplarily, the musical instrument processing workstation includes a host computer. The musical instrument processing workstation also includes a robot. The host computer can obtain the sound beam image of the sound beam to be installed corresponding to the target musical instrument located at the sound beam storage position through a 3D visual system, and perform image processing on the sound beam image based on image processing technology to obtain the current position and posture of the sound beam to be installed, that is, posture information.

[0069] For example, Figure 2 A schematic diagram of the structure of a musical instrument processing workstation provided in an embodiment of the present application, Figure 3 A schematic diagram of a musical instrument processing process provided in an embodiment of the present application, such as Figure 2 , 3As shown, the musical instrument processing workstation includes a host computer. The musical instrument processing workstation also includes a robot, a sound beam storage position, a dulcimer product, a conveyor line, a glue supply machine, a 3D vision system, and an industrial table. When the musical instrument processing process is started, the conveyor line first delivers the target musical instrument, the dulcimer, to the position, and the dulcimer is positioned on the industrial table. After reading the product information through FRID, the 3D camera of the 3D vision system is used to take a picture of the sound beam in the turnover box to collect the sound beam image and transmit it to the host computer. The host computer processes the sound beam image based on image processing technology, identifies the position of the sound beam, and obtains the coordinates and posture.

[0070] 202. According to the position information, the robot is controlled to apply glue to the sound beam to be installed.

[0071] For example, the upper computer controls the robot to perform the gluing operation on the sound beam to be installed according to the current position and posture of the sound beam to be installed, for example, Figure 2 , 3 The upper computer sends the obtained position information of the sound beam to be installed to the robot, and the robot determines the gluing route according to the position information of the sound beam to be installed, and grabs the gluing machine to perform gluing operations on the sound beam to be installed.

[0072] 203. Control the robot to install the sound beam to be installed on the target musical instrument.

[0073] For example, after determining that the robot has completed the gluing operation on the sound beam to be installed, the host computer can control the robot to install the sound beam to be installed on the target musical instrument; for example, in combination with Figure 3 , by sending a control signal to the robot to instruct the robot to perform the sound beam installation process, and after receiving the control signal, the robot installs the sound beam to be installed on the target musical instrument.

[0074] In the present embodiment, a musical instrument processing method based on image processing is provided, in which the current position and posture of the sound beam to be installed are obtained by performing image processing on the sound beam image of the sound beam to be installed corresponding to the target musical instrument, and thereby the robot is controlled to perform glue coating operation on the sound beam to be installed, and after the gluing is completed, the robot is controlled to install the sound beam to be installed on the target musical instrument; thereby, the uniform distribution of glue at both ends of the sound beam can be ensured, the bonding strength between the sound beam and the target musical instrument can be increased, and the product quality of the dulcimer can be improved.

[0075] Figure 4 A flowchart of another musical instrument processing method based on image processing provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the method includes:

[0076] 301. Obtain a sound beam image of a sound beam to be installed corresponding to a target musical instrument.

[0077] Exemplarily, this step may refer to step 201 and will not be described in detail here.

[0078] 302. Perform point cloud conversion processing on the sound beam image to obtain sound beam point cloud data corresponding to the sound beam image.

[0079] Exemplarily, the host computer performs point cloud conversion processing on the acquired sound beam image based on point cloud data processing technology. Figure 5 A schematic diagram of a point cloud conversion process provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, based on the point cloud conversion tool, the point cloud conversion parameters are set, the sound beam image is converted, and the sound beam point cloud data corresponding to the sound beam image is obtained to characterize the three-dimensional structure of the sound beam to be installed.

[0080] 303. Perform template matching processing on the sound beam point cloud data to obtain the position information of the sound beam to be installed.

[0081] Exemplarily, the host computer is based on a preset point cloud template matching rule, for example, based on a template matching tool, and can perform template matching processing on the sound beam point cloud data according to preset matching parameters to obtain the position information of the sound beam to be installed.

[0082] In one example, step 303 includes the following steps:

[0083] The first step of step 303 is to perform filtering and alignment processing on the sound beam point cloud data to obtain processed sound beam point cloud data.

[0084] The second step of step 303 is to determine the template point cloud data corresponding to the sound beam point cloud data; and determine the matching parameters.

[0085] The third step of step 303 is to match the template point cloud data and the processed sound beam point cloud data according to the matching parameters to obtain a matching result; wherein the matching result includes the position information of the sound beam to be installed.

[0086] For example, Figure 6 A schematic diagram of an image processing process provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the host computer performs filtering and alignment processing on the sound beam point cloud data based on the filtering and alignment processing technology to obtain the processed sound beam point cloud data; for example, Figure 7 A schematic diagram of a point cloud filtering process provided in an embodiment of the present application is shown in FIG. Figure 6 , 7As shown, through the point cloud processing tool, the area for 3D point cloud recognition is set, and the noise point cloud in the beam point cloud data is removed through filtering or statistical analysis, and the complexity of the point cloud is reduced through voxel grid filters or random sampling to reduce the amount of calculation. The point cloud is aligned through the ICP registration algorithm or the plane fitting algorithm to obtain the processed beam point cloud data. The upper computer processes the beam point cloud data based on the point cloud template generation tool to obtain the template point cloud data corresponding to the beam point cloud data, and determines the matching parameters, including rotation, translation, scaling, etc., as well as setting the minimum matching score, posture change range, number of matches, etc., for template matching processing. The upper computer can use the 3D template matching function, Figure 8 A schematic diagram of a template matching process provided in an embodiment of the present application is shown in FIG. Figure 6 , 8 As shown, the template point cloud data and the processed sound beam point cloud data are matched, such as feature recognition, hand-eye calibration, calibration calculation and other processing processes, to obtain a matching result, which includes the position information of the sound beam to be installed, such as matching position, rotation angle, XYZ posture direction, etc.

[0087] 304. According to the position information, the flexible grasping tool of the robot is controlled to grasp the sound beam to be installed, so as to perform a gluing operation on the sound beam to be installed.

[0088] Among them, a flexible grasping tool is arranged at the end of the robot.

[0089] For example, Fig. 9 A structural diagram of another musical instrument processing workstation provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, a flexible gripping tool is provided at the end of the robot in the musical instrument processing workstation. The upper computer sends the obtained posture information to the robot, and the robot generates a gluing operation route according to the posture information and the process requirements of the sound beam, so as to control the flexible gripping tool to grab the sound beam to be installed, and perform gluing operation on the sound beam to be installed through the gluing machine.

[0090] 305. According to the position information, control the flexible grasping tool of the robot to grasp the sound beam to be installed, so as to make the sound beam to be installed contact the soundboard of the target musical instrument.

[0091] Among them, a flexible grasping tooling is set at the end of the robot; the musical instrument processing workstation also includes a torque sensor.

[0092] For example, Fig.10 A structural diagram of another musical instrument processing workstation provided in an embodiment of the present application is shown in FIG. Fig.10As shown, a flexible gripping tool is provided at the end of the robot; the musical instrument processing workstation also includes a torque sensor; after determining that the robot has completed the gluing operation on the sound beam to be installed, the robot controls the flexible gripping tool to grab the sound beam to be installed according to the posture information of the sound beam to be installed sent by the host computer, and makes the sound beam to be installed contact the sound board of the target musical instrument.

[0093] 306. Acquire contact force information of the sound beam to be installed collected in real time by the torque sensor; wherein the contact force information represents the magnitude and direction of the contact force of the sound beam to be installed contacting the soundboard of the target musical instrument.

[0094] For example, when the robot controls the flexible grasping tooling to grasp the sound beam to be installed and contacts the soundboard of the target musical instrument, the torque sensor collects in real time the magnitude and direction of the contact force of the sound beam to be installed and contacts the soundboard of the target musical instrument, i.e., the contact force information, and transmits it to the host computer. The host computer can then obtain the contact force information of the sound beam to be installed collected in real time by the torque sensor.

[0095] 307. According to the contact force information, the robot is controlled to install the sound beam to be installed on the target musical instrument.

[0096] Exemplarily, the host computer sends the received contact force information to the robot, so that the robot installs the sound beam to be installed on the target musical instrument according to the size and direction of the contact force of the sound beam to be installed contacting the soundboard of the target musical instrument, so as to improve the accuracy of the installation of the sound beam.

[0097] In one example, step 307 includes: controlling the robot to install the sound beam to be installed on the target musical instrument according to the first contact information and the second contact information in the contact force information; wherein the first contact information represents the magnitude of the contact force in a first preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument; and the second contact information represents the magnitude of the contact force in a second preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument.

[0098] Exemplarily, the contact force information collected in real time by the torque sensor includes first contact information and second contact information, wherein the first contact information represents the magnitude of the contact force in a first preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument; and the second contact information represents the magnitude of the contact force in a second preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument. The host computer sends the acquired first contact information and second contact information to the robot, so that the robot continuously adjusts the position of the sound beam to be installed according to the first contact information and the second contact information, so as to install the sound beam to be installed on the target musical instrument.

[0099] In one example, step 307 specifically includes: if it is determined that both the first contact information and the second contact information meet the preset conditions, controlling the robot to press down and fix the sound beam to be installed on the target musical instrument.

[0100] Specifically, when the control robot installs the sound beam to be installed on the target musical instrument, when the first contact information and the second contact information collected by the upper computer both meet the preset conditions, for example, the first contact information reaches the first threshold and the second contact information reaches the second threshold, the control robot presses down and fixes the sound beam to be installed on the target musical instrument, that is, the installation of the sound beam to be installed is completed.

[0101] For example, after the gluing is completed, the robot installs the sound beam. During installation, the robot grabs the sound beam and contacts the soundboard with one end. The torque sensor senses the contact force and direction in real time. When the sound beam is installed, the torque in the Z direction is 30N-40N, and the positive and negative deviation in the X-axis direction is ±5 degrees, and the force is 15N-25N. When the torque sensor in the Z direction exceeds 40N during robot installation, the robot first lifts up the grabbed sound beam and moves it 1mm in the positive direction of the X torque, and then presses it down for installation. During installation, the Z-axis torque and X-axis torque are monitored in real time. If any of the Z-axis and X-axis forces exceed the standard value, the robot lifts up and moves 1mm in the negative direction of the torque X. This process is repeated until the Z-axis force and the X-axis force are within the standard range. The robot installs the other end of the sound beam according to this process. When the Z-axis and X-axis of the two ends of the sound beam reach the range, the robot presses the sound beam down to the specified position. In this way, the installation of a sound beam is completed.

[0102] In the present embodiment, on the basis of the above-mentioned embodiment, the image of the sound beam to be installed corresponding to the target musical instrument is processed to obtain the current position and posture of the sound beam to be installed, and this is used to control the robot to perform glue coating operation on the sound beam to be installed. After the gluing is completed, the robot is controlled to install the sound beam to be installed on the target musical instrument according to the torque sensor; on the one hand, the robot end tooling adopts a flexible clamping claw method, which can adapt the grasping stroke and the grasping force according to the product information to meet the needs of flexible production; on the other hand, the amount of glue coating can be precisely controlled to ensure the uniform distribution of glue at both ends of the sound beam, thereby improving the bonding strength between the sound beam and the target musical instrument, and thus improving the product quality of the dulcimer.

[0103] Fig.11 A schematic diagram of the structure of a musical instrument processing device based on image processing provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the device is applied to a host computer in a musical instrument processing workstation; the musical instrument processing workstation also includes a robot; the device includes:

[0104] The processing module 401 is used to obtain a sound beam image of the sound beam to be installed corresponding to the target musical instrument; and perform image processing on the sound beam image to obtain position and posture information of the sound beam to be installed; wherein the position and posture information represents the current position and posture of the sound beam to be installed;

[0105] The gluing module 402 is used to control the robot to perform gluing operation on the sound beam to be installed according to the posture information;

[0106] The installation module 403 is used to control the robot to install the sound beam to be installed on the target musical instrument.

[0107] In a possible implementation, a flexible gripping tool is provided at the end of the robot; the musical instrument processing workstation also includes a torque sensor; the installation module 403 is specifically used to: control the robot's flexible gripping tool to grip the sound beam to be installed according to the posture information, so as to make the sound beam to be installed contact the sound board of the target musical instrument; obtain the contact force information of the sound beam to be installed collected in real time by the torque sensor; wherein the contact force information represents the size and direction of the contact force of the sound beam to be installed contacting the sound board of the target musical instrument; according to the contact force information, control the robot to install the sound beam to be installed on the target musical instrument.

[0108] In a possible implementation, the installation module 403 is specifically used to: control the robot to install the sound beam to be installed on the target musical instrument according to the first contact information and the second contact information in the contact force information; wherein the first contact information represents the size of the contact force in a first preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument; and the second contact information represents the size of the contact force in a second preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument.

[0109] In a possible implementation, the installation module 403 is specifically configured to: if it is determined that both the first contact information and the second contact information satisfy a preset condition, control the robot to press down and fix the sound beam to be installed on the target musical instrument.

[0110] In a possible implementation, the processing module 401 is specifically used to: perform point cloud conversion processing on the sound beam image to obtain sound beam point cloud data corresponding to the sound beam image; perform template matching processing on the sound beam point cloud data to obtain posture information of the sound beam to be installed.

[0111] In a possible implementation, the processing module 401 is specifically used to: perform filtering and alignment processing on the sound beam point cloud data to obtain processed sound beam point cloud data; determine the template point cloud data corresponding to the sound beam point cloud data; and determine the matching parameters; according to the matching parameters, perform matching processing on the template point cloud data and the processed sound beam point cloud data to obtain a matching result; wherein the matching result includes the posture information of the sound beam to be installed.

[0112] In a possible implementation, a flexible gripping tool is provided at the end of the robot; the gluing module 402 is specifically used to: control the flexible gripping tool of the robot to grip the sound beam to be installed according to the posture information, so as to perform a gluing operation on the sound beam to be installed.

[0113] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0114] Fig.12 A schematic diagram of the structure of a host computer provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the host computer includes: a memory 501 and a processor 502; the memory 501 is a memory for storing instructions executable by the processor 502.

[0115] The processor 502 is configured to execute the method provided in the above embodiment.

[0116] The host computer further includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.

[0117] The specific implementation process of the processor can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0118] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0119] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the processing method in the above embodiment.

[0120] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the computer executes the technical solution of the processing method of the above embodiment.

[0121] An embodiment of the present application also provides a musical instrument processing workstation, which includes: a host computer and a robot, wherein the host computer stores computer execution instructions, and when the computer execution instructions are run on the computer, the computer executes the technical solution of the processing method of the above embodiment.

[0122] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0123] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist in a device as discrete components.

[0124] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the processing method in the above embodiment can be implemented.

[0125] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0126] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0127] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0128] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0129] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0130] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0132] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0133] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0134] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A musical instrument processing method based on image processing, characterized in that: The method is applied to a host computer in a musical instrument processing workstation; the musical instrument processing workstation also includes a robot; the method includes: Acquire a sound beam image of a sound beam to be installed corresponding to a target musical instrument; and perform image processing on the sound beam image to obtain position and posture information of the sound beam to be installed; wherein the position and posture information represents the current position and posture of the sound beam to be installed; According to the posture information, controlling the robot to perform a gluing operation on the sound beam to be installed; The robot is controlled to install the sound beam to be installed on the target musical instrument.

2. The method according to claim 1, characterized in that The end of the robot is provided with a flexible gripping tool; the musical instrument processing workstation also includes a torque sensor; controlling the robot to install the sound beam to be installed on the target musical instrument, including: According to the position information, controlling the flexible grasping tool of the robot to grasp the sound beam to be installed, so as to make the sound beam to be installed contact the soundboard of the target musical instrument; Acquire the contact force information of the sound beam to be installed collected in real time by the torque sensor; wherein the contact force information represents the magnitude and direction of the contact force of the sound beam to be installed contacting the soundboard of the target musical instrument; According to the contact force information, the robot is controlled to install the sound beam to be installed on the target musical instrument.

3. The method according to claim 2, characterized in that According to the contact force information, controlling the robot to install the sound beam to be installed on the target musical instrument comprises: According to the first contact information and the second contact information in the contact force information, the robot is controlled to install the sound beam to be installed on the target musical instrument; wherein the first contact information represents the magnitude of the contact force in a first preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument; and the second contact information represents the magnitude of the contact force in a second preset direction when the sound beam to be installed contacts the soundboard of the target musical instrument.

4. The method according to claim 3, characterized in that According to the first contact information and the second contact information in the contact force information, controlling the robot to install the sound beam to be installed on the target musical instrument comprises: If it is determined that both the first contact information and the second contact information satisfy a preset condition, the robot is controlled to press down and fix the sound beam to be installed on the target musical instrument.

5. The method according to claim 1, characterized in that: Performing image processing on the sound beam image to obtain the position information of the sound beam to be installed includes: Performing point cloud conversion processing on the sound beam image to obtain sound beam point cloud data corresponding to the sound beam image; Template matching is performed on the sound beam point cloud data to obtain the position information of the sound beam to be installed.

6. The method according to claim 5, characterized in that Performing template matching processing on the sound beam point cloud data to obtain the position information of the sound beam to be installed, including: Performing filtering and alignment processing on the sound beam point cloud data to obtain processed sound beam point cloud data; Determine the template point cloud data corresponding to the sound beam point cloud data; and determine the matching parameters; According to the matching parameters, the template point cloud data and the processed sound beam point cloud data are matched to obtain a matching result; wherein the matching result includes the position information of the sound beam to be installed.

7. The method according to any one of claims 1 to 6, characterized in that The end of the robot is provided with a flexible gripping tool; according to the posture information, the robot is controlled to perform a gluing operation on the sound beam to be installed, including: According to the posture information, the flexible grasping tool of the robot is controlled to grasp the sound beam to be installed, so as to perform a gluing operation on the sound beam to be installed.

8. A musical instrument processing device based on image processing, characterized in that: The device is applied to a host computer in a musical instrument processing workstation; the musical instrument processing workstation also includes a robot; the device includes: A processing module, used to obtain a sound beam image of a sound beam to be installed corresponding to a target musical instrument; and perform image processing on the sound beam image to obtain position and posture information of the sound beam to be installed; wherein the position and posture information represents a current position and posture of the sound beam to be installed; A gluing module, used for controlling the robot to perform a gluing operation on the sound beam to be installed according to the posture information; The installation module is used to control the robot to install the sound beam to be installed on the target musical instrument.

9. A host computer, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A musical instrument processing workstation, characterized in that: include: A host computer and a robot, wherein the host computer is used to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images