Scraper identification and positioning method and device based on plane camera and sensor fusion

Abstract

The invention provides a scraper identifying and positioning method and device based on fusion of a plane camera and a sensor. The method comprises the steps that firstly, a scraper stack image collected by the plane camera installed on a workpiece gripper at the position away from the center of a scraper stack area by a first fixed distance is obtained; secondly, based on an image recognition module, scraper information of a scraper to be grabbed is determined according to the scraper stack image; then, the workpiece grabbing clamp is controlled to descend, and the laser displacement sensor is controlled to measure the distance when the workpiece grabbing clamp is located at a second fixed distance position above the center point of the to-be-grabbed scraper blade according to the scraper blade information; and finally, in the descending process of the workpiece grabbing clamp, under the condition that the distance between the workpiece grabbing clamp and the scraper to be grabbed is equal to a preset numerical value, the workpiece grabbing clamp is controlled to complete the scraper grabbing task. According to the embodiment of the invention, the plane camera and the laser distance measuring sensor are combined to realize scraper positioning, so that the technical cost can be effectively reduced, and the position identification and accurate positioning of supplied scraper materials can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent manufacturing technology, and in particular to a scraper identification and positioning method and device based on the fusion of a planar camera and a sensor. Background Art

[0002] Scraper conveyor is an important industrial equipment. The loading and unloading process of its main component, the scraper, is crucial to automated production. At present, there are two main ways to load the scraper: manual and automated. However, manual operation has the problems of high labor intensity and high operating risks. For example, the scraper is prone to fall off during the lifting process, which is a safety hazard.

[0003] In this regard, the relevant technology proposes to use a smarter automated method to complete the loading of the scraper. The automated method uses a three-dimensional intelligent camera to identify the position of the scraper, which can significantly improve the safety of operation and accurately obtain the three-dimensional coordinate information of the scraper. However, the high equipment cost and complex operating procedures limit its wide application. In addition, subsequent technical maintenance and scraper model updates also face considerable challenges and costs. Therefore, it is necessary to find a smarter, more cost-effective, and safer solution to improve the efficiency and reliability of the scraper loading and unloading process. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a scraper identification and positioning method and device based on the fusion of a planar camera and a sensor, which can not only effectively reduce the technical cost, but also realize the position identification and precise positioning of the scraper incoming material.

[0005] The embodiment of the present disclosure provides a scraper identification and positioning method based on the fusion of a planar camera and a sensor, comprising:

[0006] Acquire a scraper stack image captured by a planar camera mounted on a workpiece gripper; the workpiece gripper is located at a first fixed distance above the center of the scraper stack area; the workpiece gripper is also equipped with a laser displacement sensor;

[0007] Based on the image recognition module, determine the scraper information of the scraper to be grabbed according to the scraper stack image; the scraper information includes the center point coordinates of the scraper to be grabbed;

[0008] Control the workpiece gripper to descend, and control the laser displacement sensor to measure distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be gripped according to the scraper information;

[0009] During the descending process of the workpiece gripper, when the distance between the workpiece gripper and the scraper to be gripped is equal to a preset value, the workpiece gripper is controlled to complete the scraper gripping task.

[0010] In some possible embodiments, before acquiring the scraper stack image captured by a planar camera installed on the workpiece gripper, the process includes:

[0011] Stacking scrapers of different models in their respective scraper stack areas according to preset standards, and sending parameter information corresponding to each model of scraper to the image recognition module; the parameter information includes scraper model, scraper stack layer height, scraper stack layer number, scraper number per layer, target scraper number and scraper profile information;

[0012] The center point of the region is determined based on the scraper stack region corresponding to each scraper, and the plane coordinate system is established by taking the center point of the region as the coordinate origin.

[0013] In some possible embodiments, the scraper information further includes angle information; and the image recognition module determines the scraper information of the scraper to be grabbed according to the scraper stack image, including:

[0014] Matching the scraper stack image with scraper profile information corresponding to the scraper stack based on the image recognition module, and determining a scraper grabbing order based on the matching result and the target scraper quantity;

[0015] Determine the scraper to be grabbed according to the scraper grabbing sequence, and determine the coordinates of the center point of the scraper to be grabbed based on the plane coordinate system;

[0016] It is determined whether the scraper to be grabbed has an angle deviation according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the determination result.

[0017] In some possible embodiments, determining the angle information based on the judgment result includes:

[0018] In the case where the scraper to be grabbed has an angle offset, the angle offset value is calculated according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the calculation result;

[0019] In the case that there is no angle deviation of the scraper to be grabbed, the angle information is zero.

[0020] In some possible embodiments, after determining the angle information based on the judgment result, the method includes:

[0021] The gripping angle of the workpiece gripper is adjusted according to the angle information.

[0022] In some possible embodiments, before determining the scraper grabbing order based on the matching result and the target scraper quantity, the method further includes:

[0023] Determine whether the matching result and the target scraper quantity are the same;

[0024] When the matching result and the target scraper quantity are the same, determining the scraper grabbing order according to a preset rule;

[0025] When the matching result is different from the target scraper quantity, an alarm message is issued.

[0026] In some possible embodiments, after the workpiece gripper is controlled to complete the scraper gripping task, the process includes:

[0027] The target scraper quantity corresponding to the scraper layer where the scraper to be identified is located in the image recognition module is updated based on the completion result.

[0028] The embodiment of the present disclosure provides a scraper identification and positioning device based on the fusion of a planar camera and a sensor, comprising:

[0029] An image acquisition module, used to acquire an image of a scraper stack collected by a planar camera mounted on a workpiece gripper; the workpiece gripper is located at a first fixed distance above the center of the scraper stack area; the workpiece gripper is also equipped with a laser displacement sensor;

[0030] An information determination module, used for determining the scraper information of the scraper to be grabbed according to the scraper stack image based on the image recognition module; the scraper information includes the center point coordinates of the scraper to be grabbed;

[0031] A distance measurement module, used for controlling the workpiece gripper to descend, and controlling the laser displacement sensor to measure the distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be gripped according to the scraper information;

[0032] The scraper grabbing module is used to control the workpiece grabbing clamp to complete the scraper grabbing task when the distance between the workpiece grabbing clamp and the scraper to be grabbed is equal to a preset value during the descending process of the workpiece grabbing clamp.

[0033] In some possible embodiments, the image acquisition module is further used for:

[0034] Stacking scrapers of different models in their respective scraper stack areas according to preset standards, and sending parameter information corresponding to each model of scraper to the image recognition module; the parameter information includes scraper model, scraper stack layer height, scraper stack layer number, scraper number per layer, target scraper number and scraper profile information;

[0035] The center point of the region is determined based on the scraper stack region corresponding to each scraper, and the plane coordinate system is established by taking the center point of the region as the coordinate origin.

[0036] In some possible embodiments, the scraper information further includes angle information; and the information determination module is specifically used for:

[0037] Matching the scraper stack image with scraper profile information corresponding to the scraper stack based on the image recognition module, and determining a scraper grabbing order based on the matching result and the target scraper quantity;

[0038] Determine the scraper to be grabbed according to the scraper grabbing sequence, and determine the coordinates of the center point of the scraper to be grabbed based on the plane coordinate system;

[0039] It is determined whether the scraper to be grabbed has an angle deviation according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the determination result.

[0040] In some possible embodiments, the information determination module is further used to:

[0041] In the case where the scraper to be grabbed has an angle offset, the angle offset value is calculated according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the calculation result;

[0042] In the case that there is no angle deviation of the scraper to be grabbed, the angle information is zero.

[0043] In some possible embodiments, the information determination module is further used to:

[0044] The gripping angle of the workpiece gripper is adjusted according to the angle information.

[0045] In some possible embodiments, the information determination module is further used to:

[0046] Determine whether the matching result and the target scraper quantity are the same;

[0047] When the matching result and the target scraper quantity are the same, determining the scraper grabbing order according to a preset rule;

[0048] When the matching result is different from the target scraper quantity, an alarm message is issued.

[0049] In some possible embodiments, the scraper grabbing module is also used for:

[0050] The target scraper quantity corresponding to the scraper layer where the scraper to be identified is located in the image recognition module is updated based on the completion result.

[0051] An embodiment of the present disclosure provides a computer device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, a scraper identification and positioning method based on planar camera and sensor fusion as described in any possible implementation manner described above is performed.

[0052] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the scraper identification and positioning method based on the fusion of a planar camera and a sensor as described in any possible implementation manner described above is implemented.

[0053] The scraper identification and positioning method and device based on the fusion of a plane camera and a sensor provided in the embodiment of the present disclosure installs a plane camera and a laser ranging sensor on the workpiece gripper, uses the plane camera to obtain the scraper stack image, analyzes the scraper information of the scraper to be grasped, and then uses the laser ranging sensor to realize the positioning of the scraper to be grasped. In this way, not only can the technical cost be effectively reduced, but also the position identification and precise positioning of the scraper incoming material can be realized.

[0054] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required to be cited in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A flowchart of a scraper identification and positioning method based on the fusion of a planar camera and a sensor provided in an embodiment of the present disclosure is shown;

[0057] Figure 2 A schematic diagram showing the workpiece clamping position in the scraper identification and positioning method based on the fusion of a planar camera and a sensor provided in an embodiment of the present disclosure is shown;

[0058] Figure 3 A schematic diagram of plane camera image acquisition in the scraper identification and positioning method based on plane camera and sensor fusion provided in an embodiment of the present disclosure is shown;

[0059] Figure 4 A flowchart of an image recognition method in a scraper recognition and positioning method based on planar camera and sensor fusion provided in an embodiment of the present disclosure is shown;

[0060] Figure 5 A schematic diagram showing a workpiece clamp being located at a second fixed position in a scraper identification and positioning method based on planar camera and sensor fusion provided in an embodiment of the present disclosure;

[0061] Figure 6 A schematic diagram of the structure of a scraper identification and positioning device based on the fusion of a planar camera and a sensor provided in an embodiment of the present disclosure is shown;

[0062] Figure 7 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0065] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0066] Scraper conveyor is an important industrial equipment, and the loading and unloading process of its key component, the scraper, is crucial to automated production. Research has found that the loading methods for the scraper in related technologies are mainly divided into two types: manual and automatic. However, manual operation has the problems of high labor intensity and high operating risks, especially during the hoisting process, the safety hazard of the scraper falling off is more prominent.

[0067] In response to the problems caused by manual operation, the relevant technology proposes to use a more intelligent automated method to complete the scraper loading. The automated method is to identify the position of the scraper through a three-dimensional intelligent camera and perform precise three-dimensional coordinate positioning, thereby significantly improving the safety and accuracy of the operation. This not only reduces the safety risks in manual operation, but also improves production efficiency and accuracy, which meets the needs of modern industry for efficient automated production. However, despite the many advantages of the automated method, its high equipment cost and complex operating procedures limit its widespread promotion in practical applications. In addition, subsequent technical maintenance and scraper model updates also face considerable challenges and costs.

[0068] Based on the above research, a scraper identification and positioning method and device based on the fusion of a planar camera and a sensor is provided in an embodiment of the present disclosure. First, an image of a scraper stack captured by a planar camera installed on a workpiece gripper at a first fixed distance above the center of the scraper stack area is obtained; secondly, based on the image recognition module, the scraper information of the scraper to be grabbed is determined according to the scraper stack image; then the workpiece gripper is controlled to descend, and according to the scraper information, the laser displacement sensor is controlled to measure the distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be grabbed; finally, during the descent of the workpiece gripper, when the distance between the workpiece gripper and the scraper to be grabbed is equal to a preset value, the workpiece gripper is controlled to complete the scraper grabbing task.

[0069] In the disclosed embodiment, a plane camera and a laser distance sensor are installed on the workpiece gripper, and the plane camera is used to obtain the scraper stack image to analyze the scraper information of the scraper to be grabbed, and then the laser distance sensor is used to realize the positioning of the scraper to be grabbed. In this way, using a plane camera instead of a three-dimensional smart camera in the related technology can effectively reduce the technical cost; and by combining the plane camera with the laser distance sensor, the position recognition and precise positioning of the scraper incoming material can also be realized.

[0070] To facilitate understanding of this embodiment, the execution subject of the scraper identification and positioning method based on the fusion of a plane camera and a sensor provided in the embodiment of the present disclosure is first introduced in detail. The execution subject of the scraper identification and positioning method based on the fusion of a plane camera and a sensor provided in the embodiment of the present disclosure is a computer device. The computer device can be a server. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.

[0071] The following is a detailed description of the scraper identification and positioning method based on the fusion of a planar camera and a sensor provided in the embodiment of the present application in conjunction with the accompanying drawings. Figure 1 As shown, it is a flow chart of a scraper identification and positioning method based on the fusion of a planar camera and a sensor provided by an embodiment of the present disclosure, and the method includes the following S101 to S104:

[0072] S101, acquiring a scraper stack image captured by a planar camera installed on a workpiece gripper; the workpiece gripper is located at a first fixed distance above the center of the scraper stack area; the workpiece gripper is also equipped with a laser displacement sensor.

[0073] It is understandable that the plane camera is mainly used to obtain high-resolution images on a plane. Different from the three-dimensional smart camera in the related art, it focuses on capturing two-dimensional images on a plane, rather than scenes in three-dimensional space. The plane camera is installed on the workpiece gripper in a follow-up installation mode, so as to ensure that the plane camera can move with the movement of the workpiece gripper, thereby better capturing the scraper stack image without causing image distortion or excessive changes in viewing angle.

[0074] Specifically, refer to Figure 2-3 As shown, in the embodiment of the present disclosure, the workpiece gripper A equipped with the planar camera B is 1.6 meters above the center of the scraper stack area C, completing the task of collecting the image of the scraper stack D. In some other embodiments, the distance value of the first fixed distance position can be determined according to the camera parameters of the planar camera, such as 1.2 meters or 1.8 meters, etc., which is not specifically limited here.

[0075] Here, the workpiece gripper is also equipped with a laser displacement sensor, which is a high-precision measuring device mainly used to measure tiny displacements on the surface of an object. It uses the interference principle of laser to measure distance and can provide very accurate distance data.

[0076] Exemplarily, before acquiring the scraper stack image captured by the planar camera installed on the workpiece gripper, the following (1) to (2) may be included:

[0077] (1) stacking scrapers of different models in their respective scraper stacking areas according to preset standards, and sending parameter information corresponding to each scraper model to the image recognition module;

[0078] (2) Determine the center point of the region based on the scraper stack region corresponding to each scraper, and establish a plane coordinate system using the center point of the region as the coordinate origin.

[0079] It is understandable that when scrapers of different models are put into storage, each scraper of different models can be stacked in its own fixed scraper stack area according to preset standards, and the parameter information corresponding to each model of scraper is sent to the image recognition module for storage and management. Here, the parameter information may include the scraper model, the scraper stack layer height, the number of scraper stack layers, the number of scrapers per layer, the target number of scrapers, and the scraper profile information. Among them, the target number of scrapers refers to the number of scrapers in the scraper layer to be grabbed in the scraper stack, and its initial value is equal to the number of scrapers per layer of the initially stacked scraper stack. When the workpiece gripper starts to grab the scrapers of the scraper stack, the image recognition module will update the target number of scrapers in real time according to the completion result of the grabbing task to reflect the completion status of the grabbing task and the status of the current scraper stack.

[0080] The scraper stack layer height and the number of scraper stack layers are used to determine the distance between the workpiece gripper and the scraper stack surface. Specifically, during image acquisition, the formula for determining the distance between the workpiece gripper and the scraper stack surface is expressed as:

[0081] L1=h×n+l1;

[0082] Wherein, L1 represents the distance between the workpiece gripper and the scraper stack during image acquisition; h represents the scraper stack layer height; n represents the number of scraper stack layers; l1 represents the first fixed distance.

[0083] At the same time, after the scrapers are stacked, the center point of the area is determined according to the scraper stack area, and the plane coordinate system is established with the center point of the area as the coordinate origin. The process of determining the center point of the area usually involves calculating the geometric center or center of gravity of each scraper stack area. Once the center point is determined, the plane camera can collect images based on its center point as a reference. By establishing a plane coordinate system, the position and posture of each scraper on the plane can be accurately calculated and described.

[0084] S102, based on the image recognition module, determine the scraper information of the scraper to be grabbed according to the scraper stack image; the scraper information includes the center point coordinates of the scraper to be grabbed.

[0085] It can be understood that the scraper information includes the center point coordinates and angle information of the scraper to be grabbed. After the scraper stack image is acquired, the image recognition module is used to perform image recognition on the scraper stack image to obtain the scraper information for the scraper to be grabbed. Figure 4FIG. 1 is a flowchart of an image recognition method provided by an embodiment of the present disclosure, and the method includes the following S1021 to S1023:

[0086] S1021, matching the scraper stack image with scraper profile information corresponding to the scraper stack based on the image recognition module, and determining a scraper grabbing order based on the matching result and a target scraper quantity.

[0087] Here, based on the image recognition module, the scraper profile information corresponding to the scraper stack stored in it is matched with the collected scraper stack image to obtain the number of scrapers in the layer to be grabbed in the scraper stack, and the matched number of scrapers is compared with the target number of scrapers to determine whether the matching result and the target number of scrapers are the same; if they are the same, the scraper grabbing order is determined according to preset rules (for example, grabbing from left to right or from right to left); if they are different, an alarm message is issued to notify relevant operators to check and intervene.

[0088] S1022, determining the scraper to be grabbed according to the scraper grabbing sequence, and determining the center point coordinates of the scraper to be grabbed based on the plane coordinate system.

[0089] It is understandable that after the scraper to be grabbed is determined according to the scraper grabbing order, the center point coordinates of the scraper to be grabbed are determined based on the plane coordinate system established above to ensure that the workpiece gripper can be accurately positioned at the ideal grabbing position of the scraper.

[0090] S1023, judging whether the scraper to be grabbed has an angle deviation according to the coordinates of the center point of the scraper to be grabbed, and determining angle information based on the judgment result.

[0091] Specifically, after obtaining the coordinates of the center point of the scraper to be grasped, the obtained center point coordinates are used to compare the vertical direction of the x-axis or y-axis in the plane coordinate system to determine whether the scraper to be grasped has an angle deviation. Among them, the angle deviation of the scraper may be caused by irregular material stacking or slight errors in the operation of the staff. If the scraper to be grasped has an angle deviation, the angle deviation value can be calculated based on the coordinates of the center point of the scraper to be grasped. The specific method may include calculating the deviation angle using the inverse tangent function. After determining the angle information according to the calculation result, the angle information is used to adjust the grasping angle of the workpiece gripper to ensure that the subsequent grasping operation of the workpiece gripper can grasp the scraper according to the correct angle, thereby ensuring accuracy and stability during the operation. If the scraper to be grasped does not have an angle deviation, it means that the scraper is already in the ideal grasping position, and its angle information is zero, so there is no need to adjust the grasping angle of the workpiece gripper.

[0092] In this way, the possible angle deviation of the scraper to be grasped can be effectively handled, ensuring the precise positioning and accurate grasping of the scraper during the working process, thereby improving efficiency and reliability.

[0093] S103, controlling the workpiece gripper to descend, and according to the scraper information, controlling the laser displacement sensor to measure distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be gripped.

[0094] Here, after obtaining the scraper information, as the workpiece gripper descends, the workpiece gripper is moved to above the center point of the scraper to be grabbed according to the scraper information, and the laser displacement sensor is controlled to measure the distance at a second fixed distance position above the center point of the scraper to be grabbed. The second fixed position can be specifically set according to the laser displacement sensor, for example, the value can be 0.5m, 0.8m, etc., which is not specifically set here.

[0095] For example, the method of step S101 is the same as above, and the distance of the workpiece gripping distance above the center point of the scraper to be gripped is determined by using the scraper stack height and the number of scraper stack layers, and the formula is expressed as:

[0096] L2=h×n+l2;

[0097] Among them, L2 represents the distance between the workpiece clamp and the center point of the scraper to be grasped; h represents the scraper stack height; n represents the number of scraper stack layers; l2 represents the second fixed distance.

[0098] Reference Figure 5 As shown, in the embodiment of the present disclosure, the value of the second fixed position is set to 0.5 m, that is, when the workpiece gripper A is located 0.5 m above the center point of the scraper E to be gripped, the laser displacement sensor starts to measure the distance.

[0099] S104, during the descending process of the workpiece gripper, when the distance between the workpiece gripper and the scraper to be gripped is equal to a preset value, controlling the workpiece gripper to complete the scraper gripping task.

[0100] It is understandable that during the descent of the workpiece gripper, the laser displacement sensor is used to continuously measure the distance until the distance between the workpiece gripper and the scraper to be grabbed is equal to the preset value, the workpiece gripper stops descending and triggers the corresponding grabbing action to complete the scraper grabbing task, and then updates the target scraper number corresponding to the scraper layer where the scraper to be identified is located in the image recognition module based on the completion result. Here, the preset distance can be set according to the parameter information of the workpiece gripper or the size of the scraper to be grabbed, and is not specifically limited here.

[0101] The scraper identification and positioning method and device based on the fusion of a plane camera and a sensor provided in the embodiment of the present disclosure installs a plane camera and a laser ranging sensor on the workpiece gripper, uses the plane camera to obtain the scraper stack image, analyzes the scraper information of the scraper to be grabbed, and then uses the laser ranging sensor to realize the positioning of the scraper to be grabbed. In this way, using a plane camera to replace the three-dimensional smart camera in the related technology can effectively reduce the technical cost and require less time and lower the technical threshold, which is conducive to the promotion and application of the technology; and by combining the plane camera with the laser ranging sensor, it can also realize the position identification and precise positioning of the scraper incoming material.

[0102] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0103] Based on the same inventive concept, the embodiment of the present disclosure also provides a scraper identification and positioning device based on planar camera and sensor fusion corresponding to the scraper identification and positioning method based on planar camera and sensor fusion. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned scraper identification and positioning method based on planar camera and sensor fusion in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0104] Reference Figure 6 FIG. 1 is a schematic diagram of a scraper identification and positioning device 600 based on a planar camera and a sensor fusion according to an embodiment of the present disclosure, wherein the device comprises:

[0105] An image acquisition module 601 is used to acquire a scraper stack image captured by a planar camera installed on a workpiece gripper; the workpiece gripper is located at a first fixed distance above the center of the scraper stack area; the workpiece gripper is also equipped with a laser displacement sensor;

[0106] An information determination module 602 is used to determine the scraper information of the scraper to be grabbed according to the scraper stack image based on the image recognition module; the scraper information includes the coordinates of the center point of the scraper to be grabbed;

[0107] The distance measurement module 603 is used to control the workpiece gripper to descend, and according to the scraper information, control the laser displacement sensor to measure the distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be gripped;

[0108] The scraper grabbing module 604 is used to control the workpiece gripper to complete the scraper grabbing task when the distance between the workpiece gripper and the scraper to be grabbed is equal to a preset value during the descending process of the workpiece gripper.

[0109] In some possible embodiments, the image acquisition module 601 is further used for:

[0110] Stacking scrapers of different models in their respective scraper stack areas according to preset standards, and sending parameter information corresponding to each model of scraper to the image recognition module; the parameter information includes scraper model, scraper stack layer height, scraper stack layer number, scraper number per layer, target scraper number and scraper profile information;

[0111] The center point of the region is determined based on the scraper stack region corresponding to each scraper, and the plane coordinate system is established by taking the center point of the region as the coordinate origin.

[0112] In some possible embodiments, the scraper information further includes angle information; the information determination module 602 is specifically used to:

[0113] Matching the scraper stack image with scraper profile information corresponding to the scraper stack based on the image recognition module, and determining a scraper grabbing order based on the matching result and the target scraper quantity;

[0114] Determine the scraper to be grabbed according to the scraper grabbing sequence, and determine the coordinates of the center point of the scraper to be grabbed based on the plane coordinate system;

[0115] It is determined whether the scraper to be grabbed has an angle deviation according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the determination result.

[0116] In some possible embodiments, the information determination module 602 is further configured to:

[0117] In the case where the scraper to be grabbed has an angle offset, the angle offset value is calculated according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the calculation result;

[0118] In the case that there is no angle deviation of the scraper to be grabbed, the angle information is zero.

[0119] In some possible embodiments, the information determination module 602 is further configured to:

[0120] The gripping angle of the workpiece gripper is adjusted according to the angle information.

[0121] In some possible embodiments, the information determination module 602 is further configured to:

[0122] Determine whether the matching result and the target scraper quantity are the same;

[0123] When the matching result and the target scraper quantity are the same, determining the scraper grabbing order according to a preset rule;

[0124] When the matching result is different from the target scraper quantity, an alarm message is issued.

[0125] In some possible embodiments, the scraper grabbing module 604 is further used for:

[0126] The target scraper quantity corresponding to the scraper layer where the scraper to be identified is located in the image recognition module is updated based on the completion result.

[0127] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 7 , which is a schematic diagram of the structure of a computer device 700 provided in an embodiment of the present disclosure, including a processor 701, a memory 702, and a bus 703. The memory 702 is used to store execution instructions, including a memory 7021 and an external memory 7022; the memory 7021 is also called an internal memory, which is used to temporarily store the operation data in the processor 701 and the data exchanged with the external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the memory 7021.

[0128] In the embodiment of the present application, the memory 702 is specifically used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 701. That is, when the computer device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the application code stored in the memory 702, and then executes the method described in any of the above embodiments.

[0129] Among them, the memory 702 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0130] Processor 701 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0131] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the computer device 700. In other embodiments of the present application, the computer device 700 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0132] The present disclosure also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the scraper identification and positioning method based on the fusion of a planar camera and a sensor described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0133] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the scraper identification and positioning method based on the fusion of a planar camera and a sensor described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0134] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0135] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A scraper identification and positioning method based on plane camera and sensor fusion, It is characterized in that include: Acquire a scraper stack image captured by a planar camera mounted on a workpiece gripper; the workpiece gripper is located at a first fixed distance above the center of the scraper stack area; the workpiece gripper is also equipped with a laser displacement sensor; Based on the image recognition module, determine the scraper information of the scraper to be grabbed according to the scraper stack image; the scraper information includes the center point coordinates of the scraper to be grabbed; Control the workpiece gripper to descend, and control the laser displacement sensor to measure distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be gripped according to the scraper information; During the descending process of the workpiece gripper, when the distance between the workpiece gripper and the scraper to be gripped is equal to a preset value, the workpiece gripper is controlled to complete the scraper gripping task.

2. The method according to claim 1, It is characterized in that Before obtaining the scraper stack image collected by the plane camera installed on the workpiece gripper, it includes: Stacking scrapers of different models in their respective scraper stack areas according to preset standards, and sending parameter information corresponding to each model of scraper to the image recognition module; the parameter information includes scraper model, scraper stack layer height, scraper stack layer number, scraper number per layer, target scraper number and scraper profile information; The center point of the region is determined based on the scraper stack region corresponding to each scraper, and the plane coordinate system is established by taking the center point of the region as the coordinate origin.

3. The method according to claim 2, It is characterized in that The scraper information also includes angle information; the image recognition module determines the scraper information of the scraper to be grabbed according to the scraper stack image, including: Matching the scraper stack image with scraper profile information corresponding to the scraper stack based on the image recognition module, and determining a scraper grabbing order based on the matching result and the target scraper quantity; Determine the scraper to be grabbed according to the scraper grabbing sequence, and determine the coordinates of the center point of the scraper to be grabbed based on the plane coordinate system; It is determined whether the scraper to be grabbed has an angle deviation according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the judgment result.

4. The method according to claim 3, It is characterized in that Determining the angle information based on the judgment result includes: In the case where the scraper to be grabbed has an angle offset, the angle offset value is calculated according to the coordinates of the center point of the scraper to be grabbed, and the angle information is determined based on the calculation result; In the case that there is no angle deviation of the scraper to be grabbed, the angle information is zero.

5. The method according to claim 4, It is characterized in that After determining the angle information based on the judgment result; comprising: The gripping angle of the workpiece gripper is adjusted according to the angle information.

6. The method according to claim 3, It is characterized in that Before determining the scraper grabbing order based on the matching result and the target scraper quantity, the method further includes: Determine whether the matching result and the target scraper quantity are the same; When the matching result and the target scraper quantity are the same, determining the scraper grabbing order according to a preset rule; When the matching result is different from the target scraper quantity, an alarm message is issued.

7. The method according to claim 6, It is characterized in that After the workpiece gripper is controlled to complete the scraper gripping task, the method includes: The target scraper quantity corresponding to the scraper layer where the scraper to be identified is located in the image recognition module is updated based on the completion result.

8. A scraper identification and positioning device based on plane camera and sensor fusion, It is characterized in that include: An image acquisition module, used to acquire an image of a scraper stack collected by a planar camera mounted on a workpiece gripper; the workpiece gripper is located at a first fixed distance above the center of the scraper stack area; the workpiece gripper is also equipped with a laser displacement sensor; An information determination module, used for determining the scraper information of the scraper to be grabbed according to the scraper stack image based on the image recognition module; the scraper information includes the center point coordinates of the scraper to be grabbed; A distance measurement module, used for controlling the workpiece gripper to descend, and controlling the laser displacement sensor to measure the distance when the workpiece gripper is located at a second fixed distance above the center point of the scraper to be gripped according to the scraper information; The scraper grabbing module is used to control the workpiece grabbing clamp to complete the scraper grabbing task when the distance between the workpiece grabbing clamp and the scraper to be grabbed is equal to a preset value during the descending process of the workpiece grabbing clamp.

9. A storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images