Galvanometer machining method, device and equipment and storage medium

Abstract

The invention discloses a galvanometer machining method, device and equipment and a storage medium. The galvanometer machining method, device and equipment are used for monitoring the galvanometer machining process in real time. The method comprises the following steps: acquiring an original image of a galvanometer processing area and receiving a galvanometer processing instruction; based on the galvanometer machining instruction and a preset instruction track corresponding relation, a galvanometer light emitting track corresponding to the galvanometer machining instruction is obtained; obtaining a target pixel point based on the brightness value of each pixel point in the original image; determining whether the target pixel point deviates or not based on the coordinates of the target pixel point and the light emitting track of the galvanometer; if it is determined that the target pixel point deviates, the laser is controlled to stop galvanometer machining. The target pixel point is determined based on the brightness value of the pixel point, the position of a light spot occurring in the galvanometer machining process can be positioned in real time, galvanometer machining is stopped in time when deviation occurs, real-time monitoring of the galvanometer machining process is achieved, and accidents and potential safety hazards caused by deviation of a galvanometer light emitting track are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of laser galvanometer processing, and in particular to a galvanometer processing method, device, equipment and storage medium. Background Art

[0002] With the development of technology, galvanometers are widely used in fields such as life sciences and medical diagnosis. In the process of galvanometer processing in related technologies, in actual application scenarios, the inventors found that, for example, in the process of galvanometer processing a circuit board, the galvanometer processing state is open-loop. When the galvanometer emits light abnormally, the galvanometer itself cannot detect its current state well, resulting in serious deviation of the galvanometer light trajectory, abnormal light emission at the same position and other galvanometer abnormalities. Finally, it may not only damage the circuit board and affect the production efficiency of the circuit board, but also may cause production accidents and safety hazards. Summary of the invention

[0003] In view of this, the present application provides a galvanometer processing method, device, equipment and storage medium to facilitate real-time monitoring of the galvanometer processing process.

[0004] In a first aspect, an embodiment of the present application provides a galvanometer processing method, the method comprising:

[0005] Acquire the original image of the galvanometer processing area and receive the galvanometer processing instruction;

[0006] Based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory, the galvanometer light emission trajectory corresponding to the galvanometer processing instruction is obtained;

[0007] Obtaining a target pixel based on the brightness value of each pixel in the original image;

[0008] Determine whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer;

[0009] If it is determined that the target pixel point is offset, the laser is controlled to stop the galvanometer processing.

[0010] In the present application, the position of the light spot that appears in the galvanometer processing process can be located in real time by determining the target pixel based on the brightness value of the pixel, and the galvanometer light output trajectory is used to determine whether the target pixel is offset and the galvanometer processing is stopped in time when the offset occurs, thereby realizing real-time monitoring of the galvanometer processing process and avoiding accidents and safety hazards caused by the offset of the galvanometer light output trajectory.

[0011] In some possible embodiments, determining whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer includes:

[0012] Convert the coordinates of the target pixel point from the image coordinate system to the world coordinate system;

[0013] Whether each target pixel point is offset is determined by the following steps:

[0014] Determine the pixel point to be matched from the light emission trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system;

[0015] Determine whether a first distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of the pixel point to be matched is greater than a preset distance threshold;

[0016] If it is determined that the first distance is greater than the preset distance threshold, it is determined that the target pixel point is offset;

[0017] If it is determined that the first distance is less than or equal to the preset distance threshold, it is determined that the target pixel point has not shifted.

[0018] In the present application, by matching the coordinate position of the target pixel point with the pixel point in the galvanometer light output trajectory, real-time monitoring of the galvanometer processing process is achieved, and it can be determined in time whether the target pixel point is offset.

[0019] In some possible embodiments, converting the coordinates of the target pixel point from an image coordinate system to a world coordinate system includes:

[0020] Obtaining a coordinate conversion coefficient of the original image based on a hand-eye calibration method;

[0021] Based on the coordinate conversion coefficient and the coordinate of the target pixel point in the image coordinate system, the coordinate of the target pixel point in the world coordinate system is obtained.

[0022] In the present application, the hand-eye calibration method is used to transform the coordinates of the target pixel points, thereby ensuring the accuracy of the coordinates of the target pixel points in the world coordinate system.

[0023] In some possible embodiments, determining the pixel point to be matched from the light-emitting trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system includes:

[0024] Determine a second distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of each pixel point in the galvanometer light emission trajectory;

[0025] Determine a pixel point corresponding to the smallest second distance among the second distances;

[0026] The pixel point corresponding to the smallest second distance is used as the pixel point to be matched.

[0027] In the present application, the pixel point corresponding to the smallest second distance is used as the pixel point to be matched with the target pixel point, thereby ensuring the accuracy of monitoring the galvanometer processing process.

[0028] In some possible embodiments, before receiving the galvanometer processing instruction, the method further includes:

[0029] Based on the original image of the galvanometer processing area, a plurality of galvanometer processing instructions to be selected, and a pre-trained prediction model, a plurality of galvanometer light emission trajectories corresponding to the original image are obtained;

[0030] Generate a trajectory set based on the plurality of galvanometer light emission trajectories;

[0031] In response to a user's selection operation of a galvanometer light-emitting trajectory in the trajectory set, determining a galvanometer processing instruction to be selected corresponding to the galvanometer light-emitting trajectory;

[0032] The to-be-selected galvanometer processing instruction corresponding to the galvanometer light-emitting trajectory is used as the galvanometer processing instruction.

[0033] In the present application, a pre-trained prediction model is used to obtain multiple galvanometer light emission trajectories of the original image, so that the user can understand in advance the effects corresponding to different galvanometer processing instructions, thereby improving the user experience.

[0034] In some possible embodiments, the prediction model is trained according to the following method:

[0035] Acquire an original image of the galvanometer processing area, a calibrated galvanometer light-emitting trajectory, and a galvanometer processing instruction corresponding to the calibrated galvanometer light-emitting trajectory;

[0036] Constructing a training sample set based on the original image, the calibrated galvanometer light-emitting trajectory, and the galvanometer processing instructions corresponding to the calibrated galvanometer light-emitting trajectory;

[0037] The training samples in the training sample set are input into the initial prediction model, and the initial prediction model is trained in an iterative manner until a preset convergence condition is met, and the initial prediction model after the iteration is used as the prediction model.

[0038] In the present application, a training sample set is constructed by calibrating the galvanometer light emission trajectory, so that the trained prediction model can accurately predict the galvanometer light emission trajectory corresponding to the original image of the galvanometer processing area.

[0039] In some possible embodiments, the iteration process of each round is as follows:

[0040] Inputting the original image and the calibrated galvanometer light-emitting trajectory into the training sample set to obtain the galvanometer light-emitting trajectory output by the initial prediction model;

[0041] Obtaining a loss value of the initial prediction model based on the output galvanometer light-emitting trajectory and the calibrated galvanometer light-emitting trajectory;

[0042] The model parameters of the initial prediction model are adjusted based on the loss value.

[0043] In the present application, multiple rounds of iterations are used to ensure the accuracy of the prediction model in predicting the original image of the galvanometer processing area.

[0044] In some possible embodiments, before obtaining the target pixel based on the brightness value of each pixel in the original image, the method further includes:

[0045] Performing size correction processing on the original image based on a camera distortion calibration method to obtain a corrected original image;

[0046] The step of obtaining a target pixel based on the brightness value of each pixel in the original image includes:

[0047] The target pixel is obtained based on the brightness value of each pixel in the corrected original image.

[0048] In the present application, the camera distortion calibration method is used to correct the size of the original image, thereby further ensuring the accuracy of the subsequent monitoring of the galvanometer processing process.

[0049] In some possible embodiments, the galvanometer processing method is used in the processing of a circuit board; obtaining a target pixel point based on the brightness value of each pixel point in the original image includes:

[0050] Determine the brightness value of each pixel in the original image;

[0051] For each pixel in the original image:

[0052] Determine the difference between the brightness value of the pixel point and the brightness value of each adjacent pixel point; the adjacent pixel points include pixel points having the same horizontal coordinate as the pixel point in the image coordinate system and a vertical coordinate difference equal to a preset coordinate difference, and pixel points having the same vertical coordinate as the pixel point in the image coordinate system and a horizontal coordinate difference equal to the preset coordinate difference;

[0053] If it is determined that the difference between the brightness value of the pixel point and each of the adjacent pixel points is greater than a preset difference threshold, the pixel point is determined to be the target pixel point.

[0054] In the present application, a target pixel is determined based on the difference between the brightness values ​​of the pixel and adjacent pixels, thereby ensuring the accuracy of the determined target pixel and further ensuring the accuracy of subsequent monitoring of the galvanometer processing process.

[0055] In a second aspect, an embodiment of the present application provides a galvanometer processing device, the device comprising:

[0056] An instruction acquisition module is used to acquire the original image of the galvanometer processing area and receive the galvanometer processing instructions;

[0057] A trajectory determination module, used for obtaining the galvanometer light emission trajectory corresponding to the galvanometer processing instruction based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory;

[0058] A target pixel determination module, used to obtain a target pixel based on the brightness value of each pixel in the original image;

[0059] An offset determination module, used to determine whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer;

[0060] The processing stop module is used to control the laser to stop the galvanometer processing if it is determined that the target pixel point is offset.

[0061] In some possible embodiments, when the offset determination module determines whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer, it is specifically used to:

[0062] Convert the coordinates of the target pixel point from the image coordinate system to the world coordinate system;

[0063] Whether each target pixel point is offset is determined by the following steps:

[0064] Determine the pixel point to be matched from the light emission trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system;

[0065] Determine whether a first distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of the pixel point to be matched is greater than a preset distance threshold;

[0066] If it is determined that the first distance is greater than the preset distance threshold, it is determined that the target pixel point is offset;

[0067] If it is determined that the first distance is less than or equal to the preset distance threshold, it is determined that the target pixel point has not shifted.

[0068] In some possible embodiments, when the offset determination module converts the coordinates of the target pixel point from the image coordinate system to the world coordinate system, it is specifically used to:

[0069] Obtaining a coordinate conversion coefficient of the original image based on a hand-eye calibration method;

[0070] Based on the coordinate conversion coefficient and the coordinate of the target pixel point in the image coordinate system, the coordinate of the target pixel point in the world coordinate system is obtained.

[0071] In some possible embodiments, when the offset determination module determines the pixel point to be matched from the galvanometer light emission trajectory based on the coordinates of the target pixel point in the world coordinate system, it is specifically used to:

[0072] Determine a second distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of each pixel point in the galvanometer light emission trajectory;

[0073] Determine a pixel point corresponding to the smallest second distance among the second distances;

[0074] The pixel point corresponding to the smallest second distance is used as the pixel point to be matched.

[0075] In some possible embodiments, before the instruction acquisition module receives the galvanometer processing instruction, it is also used to:

[0076] Based on the original image of the galvanometer processing area, a plurality of galvanometer processing instructions to be selected, and a pre-trained prediction model, a plurality of galvanometer light emission trajectories corresponding to the original image are obtained;

[0077] Generate a trajectory set based on the plurality of galvanometer light emission trajectories;

[0078] In response to a user's selection operation of a galvanometer light-emitting trajectory in the trajectory set, determining a galvanometer processing instruction to be selected corresponding to the galvanometer light-emitting trajectory;

[0079] The to-be-selected galvanometer processing instruction corresponding to the galvanometer light-emitting trajectory is used as the galvanometer processing instruction.

[0080] In some possible embodiments, the prediction model is trained according to the following method:

[0081] Acquire an original image of the galvanometer processing area, a calibrated galvanometer light-emitting trajectory, and a galvanometer processing instruction corresponding to the calibrated galvanometer light-emitting trajectory;

[0082] Constructing a training sample set based on the original image, the calibrated galvanometer light-emitting trajectory, and the galvanometer processing instructions corresponding to the calibrated galvanometer light-emitting trajectory;

[0083] The training samples in the training sample set are input into the initial prediction model, and the initial prediction model is trained in an iterative manner until a preset convergence condition is met, and the initial prediction model after the iteration is used as the prediction model.

[0084] In some possible embodiments, the iteration process of each round is as follows:

[0085] Inputting the original image and the calibrated galvanometer light-emitting trajectory into the training sample set to obtain the galvanometer light-emitting trajectory output by the initial prediction model;

[0086] Obtaining a loss value of the initial prediction model based on the output galvanometer light-emitting trajectory and the calibrated galvanometer light-emitting trajectory;

[0087] The model parameters of the initial prediction model are adjusted based on the loss value.

[0088] In some possible embodiments, before the target pixel point determination module obtains the target pixel point based on the brightness value of each pixel point in the original image, it is further configured to:

[0089] Performing size correction processing on the original image based on a camera distortion calibration method to obtain a corrected original image;

[0090] The step of obtaining a target pixel based on the brightness value of each pixel in the original image includes:

[0091] The target pixel is obtained based on the brightness value of each pixel in the corrected original image.

[0092] In some possible embodiments, the galvanometer processing method is used in the processing of a circuit board; when the target pixel point determination module is executed to obtain the target pixel point based on the brightness value of each pixel point in the original image, it is specifically used to:

[0093] Determine the brightness value of each pixel in the original image;

[0094] For each pixel in the original image:

[0095] Determine the difference between the brightness value of the pixel point and the brightness value of each adjacent pixel point; the adjacent pixel points include pixel points having the same horizontal coordinate as the pixel point in the image coordinate system and a vertical coordinate difference equal to a preset coordinate difference, and pixel points having the same vertical coordinate as the pixel point in the image coordinate system and a horizontal coordinate difference equal to the preset coordinate difference;

[0096] If it is determined that the difference between the brightness value of the pixel point and each of the adjacent pixel points is greater than a preset difference threshold, the pixel point is determined to be the target pixel point.

[0097] In the third aspect, another embodiment of the present application also provides an electronic device, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any method provided in the embodiment of the first aspect of the present application.

[0098] In a fourth aspect, another embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to enable a computer to execute any method provided in the embodiment of the first aspect of the present application.

[0099] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0101] Figure 1 A schematic diagram of an application scenario of a galvanometer processing method provided in an embodiment of the present application;

[0102] Figure 2 A schematic diagram of a flow chart of a galvanometer processing method provided in an embodiment of the present application;

[0103] Figure 3 A schematic diagram of a flow chart of determining a galvanometer processing instruction in a galvanometer processing method provided in an embodiment of the present application;

[0104] Figure 4 A flowchart of a training prediction model for a galvanometer processing method provided in an embodiment of the present application;

[0105] Figure 5 A schematic diagram of each round of iteration of an initial prediction model of a galvanometer processing method provided in an embodiment of the present application;

[0106] Figure 6 A schematic diagram of a correspondence relationship between preset instruction trajectories of a galvanometer processing method provided in an embodiment of the present application;

[0107] Figure 7 A schematic diagram of a process for determining a target pixel point in a galvanometer processing method provided in an embodiment of the present application;

[0108] Figure 8 A schematic diagram of the distribution of pixels in an original image of a galvanometer processing method provided in an embodiment of the present application;

[0109] Fig. 9 A schematic diagram of a flow chart of determining whether a target pixel point is offset in a galvanometer processing method provided in an embodiment of the present application;

[0110] Fig.10 A schematic diagram of a process for determining pixel points to be matched in a galvanometer processing method provided in an embodiment of the present application;

[0111] Fig.11 A schematic diagram of a device for a galvanometer processing method provided in an embodiment of the present application;

[0112] Fig.12 A schematic diagram of an electronic device for a galvanometer processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0113] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0114] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0115] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0116] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0117] The inventors have found that with the development of technology, galvanometers are widely used in fields such as life sciences and medical diagnosis. In the process of galvanometer processing in related technologies, in actual application scenarios, the inventors found that, for example, in the process of galvanometer processing a circuit board, the galvanometer processing state is open-loop. When the galvanometer emits light abnormally, the galvanometer itself cannot detect its current state well, resulting in serious deviation of the galvanometer light trajectory, abnormal light emission at the same position and other galvanometer abnormalities. Finally, not only may the circuit board be damaged and the production efficiency of the circuit board be affected, but production accidents and safety hazards may also occur.

[0118] In response to the above problems, the embodiments of the present application provide a galvanometer processing method, device, equipment and storage medium for realizing real-time monitoring of the galvanometer processing process. The inventive concept of the present application can be summarized as follows: obtaining the original image of the galvanometer processing area and receiving the galvanometer processing instruction; obtaining the galvanometer light emission trajectory corresponding to the galvanometer processing instruction based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory; obtaining the target pixel point based on the brightness value of each pixel point in the original image; determining whether the target pixel point is offset based on the coordinates of the target pixel point and the galvanometer light emission trajectory; if it is determined that the target pixel point is offset, the laser is controlled to stop the galvanometer processing. In the present application, the position of the light spot appearing in the galvanometer processing process can be located in real time by determining the target pixel point based on the brightness value of the pixel point, and the galvanometer light emission trajectory is used to determine whether the target pixel point is offset and the galvanometer processing is stopped in time when the offset occurs, thereby realizing real-time monitoring of the galvanometer processing process and avoiding the generation of accidents and safety hazards caused by the offset of the galvanometer light emission trajectory.

[0119] For ease of understanding, a galvanometer processing method provided in an embodiment of the present application is described in detail below with reference to the accompanying drawings:

[0120] like Figure 1 As shown, it is an application scenario diagram of a galvanometer processing method in an embodiment of the present application. The diagram includes: a server 10, a terminal device 20, and a camera 30; wherein:

[0121] The server 10 obtains the original image of the galvanometer processing area through the camera 30 and receives the galvanometer processing instruction through the terminal device 20; based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory, the galvanometer light emission trajectory corresponding to the galvanometer processing instruction is obtained; based on the brightness value of each pixel in the original image, the target pixel is obtained; based on the coordinates of the target pixel and the galvanometer light emission trajectory, it is determined whether the target pixel is offset; if it is determined that the target pixel is offset, the laser is controlled to stop the galvanometer processing.

[0122] The description in this application only details a single server 10, terminal device 20, and camera 30, but those skilled in the art should understand that the server 10, terminal device 20, and camera 30 shown are intended to represent the operation of the server 10, terminal device 20, and camera 30 involved in the technical solution of this application. It does not imply any limitation on the number, type, or location of the server 10, terminal device 20, and camera 30. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, the underlying concepts of the example embodiments of this application will not be changed. In addition, those skilled in the art will understand that the sending and receiving of the above data also needs to be implemented through a network.

[0123] It should be noted that the galvanometer processing method proposed in this application is not only applicable to Figure 1 The application scenario shown is also applicable to any device that requires galvanometer processing.

[0124] like Figure 2 FIG. 1 is a flow chart of a galvanometer processing method provided in an embodiment of the present application, wherein:

[0125] In step 201: an original image of a galvanometer processing area is acquired and a galvanometer processing instruction is received.

[0126] In step 202: based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory, the galvanometer light emission trajectory corresponding to the galvanometer processing instruction is obtained.

[0127] In step 203: a target pixel is obtained based on the brightness value of each pixel in the original image.

[0128] In step 204: based on the coordinates of the target pixel and the light emission trajectory of the galvanometer, it is determined whether the target pixel is offset.

[0129] In step 205: if it is determined that the target pixel point is offset, the laser is controlled to stop the galvanometer processing.

[0130] In the present application, the position of the light spot that appears in the galvanometer processing process can be located in real time by determining the target pixel based on the brightness value of the pixel, and the galvanometer light output trajectory is used to determine whether the target pixel is offset and the galvanometer processing is stopped in time when the offset occurs, thereby realizing real-time monitoring of the galvanometer processing process and avoiding accidents and safety hazards caused by the offset of the galvanometer light output trajectory.

[0131] In order to facilitate further understanding of a galvanometer processing method provided in an embodiment of the present application, Figure 2 The steps are described in detail:

[0132] In some possible embodiments, in order to facilitate the user to understand the effect diagrams corresponding to different galvanometer light emission trajectories during the galvanometer processing process, before the user selects the galvanometer processing instruction, the different galvanometer light emission trajectories corresponding to different galvanometer processing instructions are first displayed to the user, so that the user can select the galvanometer processing instruction based on the different galvanometer light emission trajectories. Specifically, it can be implemented as follows Figure 3 The steps shown, where:

[0133] In step 301: based on the original image of the galvanometer processing area, a plurality of galvanometer processing instructions to be selected, and a pre-trained prediction model, a plurality of galvanometer light emission trajectories corresponding to the original image are obtained.

[0134] In some possible embodiments, when training the prediction model, the following may be implemented: Figure 4 The steps shown, where:

[0135] In step 401: an original image of a galvanometer processing area, a calibrated galvanometer light-emitting trajectory, and a galvanometer processing instruction corresponding to the calibrated galvanometer light-emitting trajectory are obtained.

[0136] In step 402: a training sample set is constructed based on the original image, the calibrated galvanometer light-emitting trajectory, and the galvanometer processing instructions corresponding to the calibrated galvanometer light-emitting trajectory.

[0137] In the present application, in order to enable the prediction model to process the original image of the galvanometer processing area based on different galvanometer processing instructions and thus obtain different galvanometer light trajectories, a training sample set is constructed using the original image, calibrated galvanometer light trajectories, and galvanometer processing instructions corresponding to the calibrated galvanometer light trajectories.

[0138] For example: for original image 1, the galvanometer light emission trajectory generated by original image 1 based on galvanometer processing instruction 1 is trajectory 1, the galvanometer light emission trajectory generated by original image 1 based on galvanometer processing instruction 2 is trajectory 2, and the galvanometer light emission trajectory generated by original image 3 based on galvanometer processing instruction 3 is trajectory 3.

[0139] In step 403: the training samples in the training sample set are input into the initial prediction model, and the initial prediction model is trained in an iterative manner until a preset convergence condition is met, and the initial prediction model after the iteration is used as the prediction model.

[0140] In some possible embodiments, each round of iteration of the initial prediction model is as follows: Figure 5 As shown, where:

[0141] In step 501: the original image and the calibrated galvanometer light trajectory are input into the training sample set to obtain the galvanometer light trajectory output by the initial prediction model.

[0142] For example: the constructed training sample set includes: the galvanometer light emission trajectory of the original image 1 generated based on the galvanometer processing instruction 1 is trajectory 1, the galvanometer light emission trajectory of the original image 1 generated based on the galvanometer processing instruction 2 is trajectory 2, and the galvanometer light emission trajectory of the original image 3 generated based on the galvanometer processing instruction 3 is trajectory 3; the original image 1 and the galvanometer processing instruction 1 are input into the initial prediction model to obtain the galvanometer light emission trajectory 1' output by the initial prediction model; the original image 1 and the galvanometer processing instruction 2 are input into the initial prediction model to obtain the galvanometer light emission trajectory 2' output by the initial prediction model; the original image 1 and the galvanometer processing instruction 3 are input into the initial prediction model to obtain the galvanometer light emission trajectory 3' output by the initial prediction model.

[0143] In step 502: a loss value of an initial prediction model is obtained based on the outputted galvanometer light-emitting trajectory and the calibrated galvanometer light-emitting trajectory.

[0144] For example: based on the output galvanometer light output trajectory of 1', the calibrated galvanometer light output trajectory is 1, then the loss value 1 is obtained based on 1' and 1; based on the output galvanometer light output trajectory of 2', the calibrated galvanometer light output trajectory is 2, then the loss value 2 is obtained based on 2 and 2; based on the output galvanometer light output trajectory of 3', the calibrated galvanometer light output trajectory is 3, then the loss value 3 is obtained based on 3' and 3.

[0145] In step 503: the model parameters of the initial prediction model are adjusted based on the loss value.

[0146] In summary, using Figure 4 as well as Figure 5 The steps shown can obtain a trained prediction model. Through the trained prediction model, the galvanometer light output trajectory (i.e., the rendering) output by the prediction model can be obtained based on the original image and the galvanometer processing instructions required by the user, thereby avoiding the waste of resources caused by the galvanometer light output trajectory after the galvanometer processing is completed not being the galvanometer light output trajectory required by the user.

[0147] In step 302: a trajectory set is generated based on the light trajectories of multiple galvanometer mirrors.

[0148] For example: take pictures of the galvanometer processing area 1 and obtain the original image 1 corresponding to the galvanometer processing area 1; input the original image 1 and the galvanometer processing instruction 1 into the prediction model to obtain the galvanometer light output trajectory 1 output by the prediction model; input the original image 1 and the galvanometer processing instruction 2 into the prediction model to obtain the galvanometer light output trajectory 2 output by the prediction model; input the original image 1 and the galvanometer processing instruction 3 into the prediction model to obtain the galvanometer light output trajectory 3 output by the prediction model; generate a trajectory set based on the galvanometer light output trajectory 1, the galvanometer light output trajectory 2, and the galvanometer light output trajectory 3.

[0149] In step 303: in response to the user's selection operation on the galvanometer light emitting trajectory in the trajectory set, the galvanometer processing instruction to be selected corresponding to the galvanometer light emitting trajectory is determined.

[0150] In step 304: the to-be-selected galvanometer processing instruction corresponding to the galvanometer light-emitting trajectory is used as the galvanometer processing instruction.

[0151] For example, the user selects the galvanometer light emitting trajectory 3 in the trajectory set, and determines that the galvanometer processing instruction corresponding to the galvanometer light emitting trajectory 3 is the galvanometer processing instruction 3, and then the galvanometer processing instruction 3 is used as the galvanometer processing instruction.

[0152] In some possible embodiments, in order to obtain the original image of the galvanometer processing area while protecting the lens from damage, a 500W pixel color charge coupled device (CCD) camera, an anti-high-light zoom lens, and a red ring light source can be used to photograph the galvanometer processing area. It should be noted that this application only provides an embodiment, and does not limit the camera, lens, and light source used for photographing the galvanometer processing area. In specific implementation, technicians can choose the camera, lens, and light source according to their needs.

[0153] In some possible embodiments, in order to achieve real-time monitoring of the galvanometer processing process, after acquiring the original image of the galvanometer processing area and receiving the galvanometer processing instruction, the galvanometer light emission trajectory corresponding to the galvanometer processing instruction is determined based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory.

[0154] For example: The preset command trajectory correspondence relationship is as follows: Figure 6 As shown, the received galvanometer processing instruction is instruction 1, then based on Figure 6 The preset instruction trajectory correspondence shown can determine that the galvanometer light emission trajectory corresponding to the galvanometer processing instruction 1 is trajectory 1.

[0155] In some possible embodiments, after determining the galvanometer light trajectory corresponding to the galvanometer processing instruction, it is necessary to obtain the target pixel point, that is, the light spot in the original image, based on the brightness value of each pixel point in the original image. In order to ensure the accuracy of the determined target pixel point, the original image can be size-corrected based on the camera distortion calibration method to obtain the corrected original image; first determine the brightness value of each pixel point in the corrected original image, and for each pixel point in the corrected original image, it can be implemented as follows: Figure 7 The steps shown, where:

[0156] In step 701: determine the difference between the brightness value of a pixel and the brightness value of each adjacent pixel.

[0157] Wherein, the adjacent pixel points include pixel points whose horizontal coordinates are the same as those of the pixel points in the image coordinate system and whose vertical coordinate difference is equal to the preset coordinate difference, and pixel points whose vertical coordinates are the same as those of the pixel points in the image coordinate system and whose horizontal coordinate difference is equal to the preset coordinate difference;

[0158] In step 702: if the difference between the brightness value of the pixel point and each adjacent pixel point is greater than a preset difference threshold, the pixel point is determined to be a target pixel point.

[0159] For example: The distribution of pixels in the original image is as follows: Figure 8 As shown, the preset coordinate difference is 1, then for pixel 1, the adjacent pixels of pixel 1 are determined to be: pixel 2, pixel 5, and for pixel 6, the adjacent pixels of pixel 6 are determined to be: pixel 2, pixel 5, pixel 7, pixel 10. If the difference between the brightness values ​​of pixel 1 and the adjacent pixels is greater than the preset difference threshold, pixel 1 is determined to be the target pixel.

[0160] In some possible embodiments, after determining the target pixel point, it is necessary to determine whether the target pixel point is offset based on the coordinates of the target pixel point and the galvanometer light trajectory. First, the coordinates of the target pixel point are converted from the image coordinate system to the world coordinate system. Specifically, it can be implemented as follows: based on the hand-eye calibration method, the coordinate conversion coefficient of the original image is obtained; based on the coordinate conversion coefficient and the coordinates of the target pixel point in the image coordinate system, the coordinates of the target pixel point in the world coordinate system are obtained.

[0161] After the coordinates of the target pixel points are converted from the image coordinate system to the world coordinate system, for each of the target pixel points, the following specific implementation can be performed: Fig. 9 The steps shown, where:

[0162] In step 901: based on the coordinates of the target pixel in the world coordinate system, the pixel to be matched is determined from the light trajectory of the galvanometer.

[0163] In some possible embodiments, when the pixel point to be matched is determined from the light-emitting trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system, it can be specifically implemented as follows: Fig.10 The steps shown, where:

[0164] In step 1001: determine a second distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of each pixel point in the galvanometer light emission trajectory.

[0165] In step 1002: determine the pixel point corresponding to the smallest second distance among the second distances.

[0166] In step 1003: the pixel point corresponding to the smallest second distance is used as the pixel point to be matched.

[0167] For example: the galvanometer light trajectory includes 5 pixels, namely pixel A, pixel B, pixel C, pixel D, and pixel E. The second distance between the target pixel 1 and pixel A is determined to be 5, the second distance between the target pixel 1 and pixel B is determined to be 4, the second distance between the target pixel 1 and pixel C is determined to be 3, the second distance between the target pixel 1 and pixel D is determined to be 2, and the second distance between the target pixel 1 and pixel E is determined to be 1. Then the pixel to be matched is determined to be pixel E.

[0168] In step 902: determine whether the first distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of the pixel point to be matched is greater than a preset distance threshold; if so, proceed to step 903, otherwise proceed to step 904;

[0169] In step 903: determining that the target pixel point is offset;

[0170] In step 904: determine that the target pixel point has not shifted.

[0171] Continuing with the above example, it is determined that the first distance between the to-be-matched pixel point E and the target pixel point 1 is 1, and the preset distance threshold is 0.5. If it is determined that the first distance is greater than the preset distance threshold, it is determined that the target pixel point 1 is offset.

[0172] To summarize, in the present application, the position of the light spot that appears in the galvanometer processing process can be located in real time by determining the target pixel based on the brightness value of the pixel point, and whether the target pixel point is offset is determined by the galvanometer light emission trajectory and the galvanometer processing is stopped in time when an offset occurs, thereby realizing real-time monitoring of the galvanometer processing process, avoiding accidents and safety hazards caused by the offset of the galvanometer light emission trajectory, and in the present application, there is no need for human intervention in the galvanometer processing process, reducing the misjudgment caused by subjective judgment in the monitoring of the galvanometer processing process.

[0173] Based on the same inventive concept, the present application also provides a galvanometer processing device 1100, such as Fig.11 As shown, the device comprises:

[0174] The instruction acquisition module 11001 is used to acquire the original image of the galvanometer processing area and receive the galvanometer processing instruction;

[0175] The trajectory determination module 11002 is used to obtain the galvanometer light emission trajectory corresponding to the galvanometer processing instruction based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory;

[0176] A target pixel determination module 11003 is used to obtain a target pixel based on the brightness value of each pixel in the original image;

[0177] An offset determination module 11004 is used to determine whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer;

[0178] The processing stop module 11005 is used to control the laser to stop the galvanometer processing if it is determined that the target pixel point is offset.

[0179] In some possible embodiments, when the offset determination module 11004 determines whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer, it is specifically used to:

[0180] Convert the coordinates of the target pixel point from the image coordinate system to the world coordinate system;

[0181] For each of the target pixels:

[0182] Determine the pixel point to be matched from the light emission trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system;

[0183] Determine whether a first distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of the pixel point to be matched is greater than a preset distance threshold;

[0184] If it is determined that the first distance is greater than a preset distance threshold, then it is determined that the target pixel point is offset;

[0185] If it is determined that the first distance is less than or equal to the preset distance threshold, it is determined that the target pixel point has not shifted.

[0186] In some possible embodiments, when the offset determination module 11004 converts the coordinates of the target pixel point from the image coordinate system to the world coordinate system, it is specifically used to:

[0187] Obtaining a coordinate conversion coefficient of the original image based on a hand-eye calibration method;

[0188] Based on the coordinate conversion coefficient and the coordinate of the target pixel point in the image coordinate system, the coordinate of the target pixel point in the world coordinate system is obtained.

[0189] In some possible embodiments, when the offset determination module 11004 determines the pixel point to be matched from the galvanometer light emission trajectory based on the coordinates of the target pixel point in the world coordinate system, it is specifically used to:

[0190] Determine a second distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of each pixel point in the galvanometer light emission trajectory;

[0191] Determine a pixel point corresponding to the smallest second distance among the second distances;

[0192] The pixel point corresponding to the smallest second distance is used as the pixel point to be matched.

[0193] In some possible embodiments, before the instruction acquisition module 11001 receives the galvanometer processing instruction, it is also used to:

[0194] Based on the original image of the galvanometer processing area, a plurality of galvanometer processing instructions to be selected, and a pre-trained prediction model, a plurality of galvanometer light emission trajectories corresponding to the original image are obtained;

[0195] Generate a trajectory set based on the plurality of galvanometer light emission trajectories;

[0196] In response to a user's selection operation of a galvanometer light-emitting trajectory in the trajectory set, determining a galvanometer processing instruction to be selected corresponding to the galvanometer light-emitting trajectory;

[0197] The to-be-selected galvanometer processing instruction corresponding to the galvanometer light-emitting trajectory is used as the galvanometer processing instruction.

[0198] In some possible embodiments, the prediction model is trained according to the following method:

[0199] Acquire an original image of the galvanometer processing area, a calibrated galvanometer light-emitting trajectory, and a galvanometer processing instruction corresponding to the calibrated galvanometer light-emitting trajectory;

[0200] Constructing a training sample set based on the original image, the calibrated galvanometer light-emitting trajectory, and the galvanometer processing instructions corresponding to the calibrated galvanometer light-emitting trajectory;

[0201] The training samples in the training sample set are input into the initial prediction model, and the initial prediction model is trained in an iterative manner until a preset convergence condition is met, and the initial prediction model after the iteration is used as the prediction model.

[0202] In some possible embodiments, each round of iteration process is as follows:

[0203] Inputting the original image and the calibrated galvanometer light-emitting trajectory into the training sample set to obtain the galvanometer light-emitting trajectory output by the initial prediction model;

[0204] Obtaining a loss value of the initial prediction model based on the output galvanometer light-emitting trajectory and the calibrated galvanometer light-emitting trajectory;

[0205] The model parameters of the initial prediction model are adjusted based on the loss value.

[0206] In some possible embodiments, before the target pixel determination module 11003 obtains the target pixel based on the brightness value of each pixel in the original image, it is further configured to:

[0207] Performing size correction processing on the original image based on a camera distortion calibration method to obtain a corrected original image;

[0208] The step of obtaining a target pixel based on the brightness value of each pixel in the original image includes:

[0209] The target pixel is obtained based on the brightness value of each pixel in the corrected original image.

[0210] In some possible embodiments, when the target pixel determination module 11003 obtains the target pixel based on the brightness value of each pixel in the original image, it is specifically used to:

[0211] Determine the brightness value of each pixel in the original image;

[0212] For each pixel in the original image:

[0213] Determine the difference between the brightness value of the pixel point and the brightness value of each adjacent pixel point; the adjacent pixel points include pixel points having the same horizontal coordinate as the pixel point in the image coordinate system and a vertical coordinate difference equal to a preset coordinate difference, and pixel points having the same vertical coordinate as the pixel point in the image coordinate system and a horizontal coordinate difference equal to the preset coordinate difference;

[0214] If it is determined that the difference between the brightness value of the pixel point and each of the adjacent pixel points is greater than a preset difference threshold, the pixel point is determined to be a target pixel point.

[0215] Corresponding to the above embodiments, the present application also provides an electronic device. Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, the electronic device 1200 may include: a processor 1201, a memory 1202 and a communication unit 1203. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present invention, it can be a bus structure or a star structure, and can also include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0216] The communication unit 1203 is used to establish a communication channel so that the electronic device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0217] The processor 1201 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 1202, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 1201 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0218] The memory 1202 is used to store the execution instructions of the processor 1201. The memory 1202 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0219] When the execution instructions in the memory 1202 are executed by the processor 1201, the electronic device 1200 can execute Figure 7 Some or all of the steps in the illustrated embodiments.

[0220] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the calling method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0221] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0222] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

Claims

1. A galvanometer processing method, characterized in that: The method comprises: Acquire the original image of the galvanometer processing area and receive the galvanometer processing instruction; Based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory, the galvanometer light emission trajectory corresponding to the galvanometer processing instruction is obtained; Obtaining a target pixel based on the brightness value of each pixel in the original image; Determine whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer; If it is determined that the target pixel point is offset, the laser is controlled to stop the galvanometer processing.

2. The method according to claim 1, characterized in that The determining whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer includes: Convert the coordinates of the target pixel point from the image coordinate system to the world coordinate system; Whether there is an offset in each of the target pixels is determined by the following steps: Determine the pixel point to be matched from the light emission trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system; Determine whether a first distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of the pixel point to be matched is greater than a preset distance threshold; If it is determined that the first distance is greater than the preset distance threshold, it is determined that the target pixel point is offset; If it is determined that the first distance is less than or equal to the preset distance threshold, it is determined that the target pixel point has not shifted.

3. The method according to claim 2, characterized in that The step of converting the coordinates of the target pixel point from the image coordinate system to the world coordinate system comprises: Obtaining a coordinate conversion coefficient of the original image based on a hand-eye calibration method; Based on the coordinate conversion coefficient and the coordinate of the target pixel point in the image coordinate system, the coordinate of the target pixel point in the world coordinate system is obtained.

4. The method according to claim 2, characterized in that: The step of determining the pixel point to be matched from the light emission trajectory of the galvanometer based on the coordinates of the target pixel point in the world coordinate system includes: Determine a second distance between the coordinates of the target pixel point in the world coordinate system and the coordinates of each pixel point in the galvanometer light emission trajectory; Determine a pixel point corresponding to the smallest second distance among the second distances; The pixel point corresponding to the smallest second distance is used as the pixel point to be matched.

5. The method according to claim 1, characterized in that Before receiving the galvanometer processing instruction, the method further includes: Based on the original image of the galvanometer processing area, a plurality of galvanometer processing instructions to be selected, and a pre-trained prediction model, a plurality of galvanometer light emission trajectories corresponding to the original image are obtained; Generate a trajectory set based on the plurality of galvanometer light emission trajectories; In response to a user's selection operation of a galvanometer light-emitting trajectory in the trajectory set, determining a galvanometer processing instruction to be selected corresponding to the galvanometer light-emitting trajectory; The to-be-selected galvanometer processing instruction corresponding to the galvanometer light-emitting trajectory is used as the galvanometer processing instruction.

6. The method according to claim 5, characterized in that The prediction model is trained according to the following method: Acquire an original image of the galvanometer processing area, a calibrated galvanometer light-emitting trajectory, and a galvanometer processing instruction corresponding to the calibrated galvanometer light-emitting trajectory; Constructing a training sample set based on the original image, the calibrated galvanometer light-emitting trajectory, and the galvanometer processing instructions corresponding to the calibrated galvanometer light-emitting trajectory; The training samples in the training sample set are input into the initial prediction model, and the initial prediction model is trained in an iterative manner until a preset convergence condition is met, and the initial prediction model after the iteration is used as the prediction model.

7. The method according to claim 6, characterized in that The iteration process of each round is as follows: Inputting the original image and the calibrated galvanometer light-emitting trajectory into the training sample set to obtain the galvanometer light-emitting trajectory output by the initial prediction model; Obtaining a loss value of the initial prediction model based on the output galvanometer light-emitting trajectory and the calibrated galvanometer light-emitting trajectory; The model parameters of the initial prediction model are adjusted based on the loss value.

8. The method according to claim 1, characterized in that Before obtaining the target pixel based on the brightness value of each pixel in the original image, the method further includes: Performing size correction processing on the original image based on a camera distortion calibration method to obtain a corrected original image; The step of obtaining a target pixel based on the brightness value of each pixel in the original image includes: The target pixel is obtained based on the brightness value of each pixel in the corrected original image.

9. The method according to any one of claims 1 to 8, characterized in that: The galvanometer processing method is used in the processing of circuit boards; the target pixel point is obtained based on the brightness value of each pixel point in the original image, including: Determine the brightness value of each pixel in the original image; For each pixel in the original image: Determine the difference between the brightness value of the pixel point and the brightness value of each adjacent pixel point; the adjacent pixel points include pixel points having the same horizontal coordinate as the pixel point in the image coordinate system and a vertical coordinate difference equal to a preset coordinate difference, and pixel points having the same vertical coordinate as the pixel point in the image coordinate system and a horizontal coordinate difference equal to the preset coordinate difference; If it is determined that the difference between the brightness value of the pixel point and each of the adjacent pixel points is greater than a preset difference threshold, the pixel point is determined to be the target pixel point.

10. A galvanometer processing device, characterized in that: The device comprises: An instruction acquisition module is used to acquire the original image of the galvanometer processing area and receive the galvanometer processing instructions; A trajectory determination module, used for obtaining the galvanometer light emission trajectory corresponding to the galvanometer processing instruction based on the correspondence between the galvanometer processing instruction and the preset instruction trajectory; A target pixel determination module, used to obtain a target pixel based on the brightness value of each pixel in the original image; An offset determination module, used to determine whether the target pixel point is offset based on the coordinates of the target pixel point and the light emission trajectory of the galvanometer; The processing stop module is used to control the laser to stop the galvanometer processing if it is determined that the target pixel point is offset.

11. An electronic device, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 9.

Images