Operation Monitoring System and Method for Floor Tile Laying Robot Based on Machine Vision

Abstract

The embodiment of the present application provides a running monitoring system and method for a floor tile laying robot based on machine vision, belonging to the technical field of robot running control. The present application collects pictures of the bottom of floor tiles, analyzes the influence of floor tile defects on the laying flatness, collects data of tile adhesive, analyzes the influence of the overflow amount of tile adhesive on the laying flatness, collects the running data of the floor tile laying robot, analyzes the influence of the running state of the floor tile laying robot on the laying flatness, analyzes the comprehensive flatness of floor tile laying based on the influence of floor tile defects on the laying flatness, the influence of the overflow amount of tile adhesive on the laying flatness and the influence of the running state of the floor tile laying robot on the laying flatness, and analyzes whether the floor tiles meet the laying standards based on the comprehensive flatness. By monitoring floor tile defects, tile adhesive overflow and robot running during floor tile laying, the present application comprehensively estimates the flatness of floor tile laying, improves the accuracy of the flatness estimation of floor tile laying effect, and makes the laying work intelligent and standardized.

Description

Technical Field

[0001] The present application relates to the technical field of robot operation control, in particular to an operation monitoring system and method for a floor tile laying robot based on machine vision. Background Art

[0002] A floor tile laying robot is a high-tech automation device mainly used for laying floor tiles in floor decoration projects. Through laser navigation technology, visual recognition technology, and elevation positioning system, it can achieve automatic walking, precise movement, and autonomous paving, and complete the integrated operation of tile adhesive laying, floor tile transportation, and floor tile laying construction. It is widely used in residential buildings, high-speed railway stations, airports, office buildings, schools and other scenarios. The floor tile laying robot can improve construction efficiency, reduce labor intensity, and ensure paving quality.

[0003] In the prior art, the operation monitoring of the floor tile laying robot usually collects deflection data by laser, and then adjusts the working arm posture of the paving robot in multiple directions to determine the laying angle and positioning during the paving process. However, in practical applications, due to the combined influence of floor tile production defects, tile adhesive overflow, and robot operation status on the laying quality, it is impossible to accurately estimate the effect of floor tile laying only based on the analysis of the paving position and paving angle. Therefore, how to improve the accuracy of floor tile laying estimation has become a technical problem to be solved currently.

[0004] To solve the above problems, the present application provides an operation monitoring system and method for a floor tile laying robot based on machine vision. Summary of the Invention

[0005] In view of the above problems, the purpose of the present application is to provide an operation monitoring system and method for a floor tile laying robot based on machine vision. By monitoring floor tile defects, tile adhesive overflow, and robot operation during floor tile laying, the present application can timely analyze paving anomalies, comprehensively estimate the flatness of floor tile laying, improve the accuracy of floor tile laying effect estimation, and make the paving work intelligent and standardized. The specific solutions are as follows:

[0006] In the first aspect, the present application provides an operation monitoring method for a floor tile laying robot based on machine vision, including the following specific steps:

[0007] Step 1: Collect pictures of the bottom of the floor tile and analyze the influence of floor tile defects on paving flatness;

[0008] Step 2: Collect tile adhesive data and analyze the influence of tile adhesive overflow on paving flatness;

[0009] Step 3: Collect operation data of the floor tile laying robot and analyze the influence of the operation status of the floor tile laying robot on paving flatness;

[0010] Step 4: Analyze the comprehensive flatness of floor tile laying based on the influence of floor tile defects on laying flatness, the influence of the amount of tile adhesive overflow on laying flatness, and the running state of the floor tile laying robot on laying flatness;

[0011] Step 5: Analyze whether the floor tiles meet the laying standards based on the comprehensive flatness.

[0012] Optionally, the said Step 1 includes the following specific steps:

[0013] Step S11: Collect the bottom image of the floor tile as the comparison image, obtain the pre-collected standard bottom image of the floor tile, and output the comparison image and the standard bottom image of the floor tile in the form of a pixel value set;

[0014] Step S12: Calculate the floor tile defect value based on the comparison image and the standard bottom image of the floor tile. The floor tile defect value is calculated by the following formula: , where represents the floor tile defect value of the i-th floor tile, represents the pixel value set of the comparison image corresponding to the i-th floor tile, represents the pixel value set of the standard bottom image of the floor tile, represents the intersection, represents the union;

[0015] Step S13: Calculate the influence value of floor tile defects on laying flatness based on the floor tile defect value. The influence value of floor tile defects on laying flatness is calculated by the following formula: , where represents the influence value of floor tile defects on laying flatness, represents the number of floor tiles.

[0016] Optionally, the said Step 2 includes the following specific steps:

[0017] Step S21: Collect tile adhesive data, where the tile adhesive data includes the fluidity of the tile adhesive, and obtain the weight of the floor tile;

[0018] Step S22: Divide the tile adhesive fluidity and floor tile weight data sets into an 80% training data set and a 20% test data set. Input the 80% training data set into the tile adhesive overflow neural network analysis model for training and obtain the initial tile adhesive overflow neural network analysis model. Then input the 20% test data set into the initial tile adhesive overflow neural network analysis model for testing to obtain the tile adhesive overflow neural network analysis model with the highest analysis accuracy for the tile adhesive overflow amount;

[0019] Step S23: Obtain the tile adhesive overflow amount based on the neural network analysis model for tile adhesive overflow amount. The neural network analysis model for tile adhesive overflow amount includes the output strategy formula of specific neurons. The output strategy formula of the specific neurons is as follows: , where represents the output of the h-th neuron in the (j + 1)-th layer of the neural network analysis model for tile adhesive overflow amount, represents the connection weight between the r-th neuron in the j-th layer and the h-th neuron in the (j + 1)-th layer of the neural network analysis model for tile adhesive overflow amount, represents the output of the r-th neuron in the j-th layer of the neural network analysis model for tile adhesive overflow amount, represents the bias of the linear relationship between the r-th neuron in the j-th layer and the h-th neuron in the (j + 1)-th layer of the neural network analysis model for tile adhesive overflow amount, represents the Sigmoid activation function;

[0020] Step S24: Calculate the influence value of the tile adhesive overflow amount on the laying flatness based on the tile adhesive overflow amount. The influence value of the tile adhesive overflow amount on the laying flatness is calculated by the following formula: , where represents the influence value of the tile adhesive overflow amount on the laying flatness, represents the tile adhesive overflow amount corresponding to the laying of the i-th floor tile, represents the safety threshold of the tile adhesive overflow amount.

[0021] Optionally, step three includes the following specific steps:

[0022] Step S31: Collect the operation data of the floor tile laying robot. The operation data of the floor tile laying robot includes the vibration amplitude and vibration frequency;

[0023] Step S32: Calculate the influence value of the robot operation on the laying flatness based on the vibration amplitude and vibration frequency. The influence value of the robot operation on the laying flatness is calculated by the following formula: , where represents the influence value of the robot operation on the laying flatness, represents the monitoring duration of the robot operation, represents the vibration frequency of the robot operation at time t, represents the standard vibration frequency of the robot operation, represents the vibration amplitude of the robot operation at time t, represents the standard vibration amplitude of the robot operation, represents the time integral.

[0024] Optionally, step four includes the following specific steps:

[0025] Calculate the comprehensive flatness of floor tile paving based on the influence value of floor tile defects on paving flatness, the influence value of tile adhesive overflow on paving flatness, and the influence value of robot operation on paving flatness. The comprehensive flatness of floor tile paving is calculated by the following formula: , where represents the comprehensive flatness, represents the proportion coefficient of floor tile defects, represents the proportion coefficient of tile adhesive overflow, represents the proportion coefficient of robot operation.

[0026] Optionally, step five includes the following specific steps:

[0027] Analyze whether the floor tile meets the paving standard based on the comparison result between the comprehensive flatness and the preset comprehensive flatness standard value. If the comprehensive flatness is greater than or equal to the preset comprehensive flatness standard value, output that the floor tile meets the paving standard. If the comprehensive flatness is less than the preset comprehensive flatness standard value, output that the floor tile does not meet the paving standard.

[0028] In a second aspect, the present application provides a running monitoring system for a floor tile paving robot based on machine vision, which is used to implement the running monitoring method of the floor tile paving robot based on machine vision. It includes a vision acquisition module for acquiring the bottom image of the floor tile as a comparison image and the bottom image of the standard floor tile, and outputting the comparison image and the bottom image of the standard floor tile in the form of a pixel value set;

[0029] A floor tile defect influence analysis module for calculating the floor tile defect value based on the comparison image and the bottom image of the standard floor tile, and calculating the influence value of the floor tile defect on the paving flatness based on the floor tile defect value;

[0030] A data acquisition module for acquiring tile adhesive data, where the tile adhesive data includes the fluidity of the tile adhesive, acquiring the weight of the floor tile, and acquiring the running data of the floor tile paving robot, where the running data of the floor tile paving robot includes the vibration amplitude and the vibration frequency;

[0031] A tile adhesive overflow influence analysis module for training a tile adhesive overflow amount neural network analysis model based on the fluidity of the tile adhesive and the floor tile weight data set, obtaining the tile adhesive overflow amount based on the tile adhesive overflow amount neural network analysis model, and calculating the influence value of the tile adhesive overflow amount on the paving flatness based on the tile adhesive overflow amount;

[0032] A robot operation influence analysis module for calculating the influence value of robot operation on the paving flatness based on the vibration amplitude and the vibration frequency;

[0033] A comprehensive flatness analysis module for calculating the comprehensive flatness of floor tile paving based on the influence value of floor tile defects on paving flatness, the influence value of tile adhesive overflow on paving flatness, and the influence value of robot operation on paving flatness;

[0034] A standard judgment module is used to analyze whether the floor tiles meet the laying standards based on the comparison result between the comprehensive flatness and a preset comprehensive flatness standard value. If the comprehensive flatness is greater than or equal to the preset comprehensive flatness standard value, it outputs that the floor tiles meet the laying standards; if the comprehensive flatness is less than the preset comprehensive flatness standard value, it outputs that the floor tiles do not meet the laying standards.

[0035] In a third aspect, the present application provides a computer-readable storage medium, including computer operation instructions. When the computer operation instructions run on a computer, the computer is enabled to execute the above-mentioned method for monitoring the operation of a floor tile laying robot based on machine vision.

[0036] Compared with the prior art, the present application has the following beneficial effects:

[0037] The present application collects pictures of the bottom of floor tiles, analyzes the influence of floor tile defects on the laying flatness, collects tile adhesive data, analyzes the influence of the overflow amount of tile adhesive on the laying flatness, collects the operation data of the floor tile laying robot, analyzes the influence of the operation state of the floor tile laying robot on the laying flatness, analyzes the comprehensive flatness of floor tile laying based on the influence of floor tile defects on the laying flatness, the influence of the overflow amount of tile adhesive on the laying flatness, and the influence of the operation state of the floor tile laying robot on the laying flatness, and analyzes whether the floor tiles meet the laying standards based on the comprehensive flatness. By monitoring floor tile defects, tile adhesive overflow, and robot operation during floor tile laying, the present application can timely analyze laying abnormalities, comprehensively estimate the flatness of floor tile laying, improve the accuracy of estimating the effect of floor tile laying, and make the laying work intelligent and standardized. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for description in the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0039] Figure 1 It is a schematic flow chart of the method for monitoring the operation of a floor tile laying robot based on machine vision according to the present application;

[0040] Figure 2 It is a schematic diagram of step one of the method for monitoring the operation of a floor tile laying robot based on machine vision according to the present application;

[0041] Figure 3 It is a schematic diagram of step two of the method for monitoring the operation of a floor tile laying robot based on machine vision according to the present application;

[0042] Figure 4Schematic diagram of step 3 of the operation monitoring method of the floor tile laying robot based on machine vision in this application;

[0043] Figure 5 Schematic diagram of the overall framework of the operation monitoring system of the floor tile laying robot based on machine vision in this application. Specific implementation manners

[0044] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0045] The terms "first", "second", "third", etc. in the specification, claims and above-mentioned drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0046] By analyzing the background technology, it can be known that in actual applications, since the laying quality is comprehensively affected by floor tile production defects, tile adhesive overflow, and the operating state of the robot, it is impossible to accurately estimate the effect of floor tile laying only based on the analysis of the laying position and laying angle. Therefore, how to improve the accuracy of floor tile laying estimation has become a technical problem to be solved currently.

[0047] In view of the above problems, the embodiments of the present application provide an operation monitoring method for a floor tile laying robot based on machine vision, as Figure 1 shown. The operation monitoring method for the floor tile laying robot based on machine vision includes:

[0048] Step 1: Collect pictures of the bottom of the floor tile and analyze the influence of floor tile defects on the laying flatness;

[0049] As Figure 2 shown, in this embodiment, step 1 includes the following specific steps:

[0050] Step S11: Set the picture of the bottom of the floor tile collected as a comparison image, obtain the pre-collected standard picture of the bottom of the floor tile, and output the comparison image and the standard picture of the bottom of the floor tile in the form of a set of pixel values;

[0051] In specific implementation, before work, standard floor tiles without production defects are selected. The floor tile laying robot uses a vision system to collect the bottom image of the standard floor tile as a reference. During work, when the floor tile laying robot picks up a floor tile using the vision system, it collects the bottom image of the floor tile, and uses an image processing tool to traverse each pixel of the image, obtains each pixel value of the bottom image of the standard floor tile and the bottom image of the floor tile, and stores them in a set.

[0052] Step S12: Calculate the floor tile defect value based on the comparison image and the bottom image of the standard floor tile. The floor tile defect value is calculated by the following formula: , where, represents the floor tile defect value of the i-th floor tile. In this embodiment, the same pixel points of the comparison image and the bottom image of the standard floor tile are obtained through the intersection of the pixel value set of the comparison image and the pixel value set of the bottom image of the standard floor tile, and the sum of the pixel points of the comparison image and the bottom image of the standard floor tile is obtained through the union of the pixel value set of the comparison image and the pixel value set of the bottom image of the standard floor tile. When the sum of the pixel points of the comparison image and the bottom image of the standard floor tile is determined, the more the same pixel points, the more similar the comparison image and the bottom image of the standard floor tile, and the smaller the corresponding floor tile defect value. represents the pixel value set of the comparison image corresponding to the i-th floor tile, represents the pixel value set of the bottom image of the standard floor tile, represents the intersection, represents the union;

[0053] Step S13: Calculate the influence value of the floor tile defect on the laying flatness based on the floor tile defect value. The influence value of the floor tile defect on the laying flatness is calculated by the following formula: , where, represents the influence value of the floor tile defect on the laying flatness. In this embodiment, the influence value of the floor tile defect on the laying flatness is reflected by the degree of dispersion of the floor tile flatness. The higher the degree of dispersion of the floor tile flatness, the greater the influence value of the floor tile defect on the laying flatness. represents the number of floor tiles.

[0054] Step Two: Collect tile adhesive data and analyze the influence of the tile adhesive overflow amount on the laying flatness;

[0055] As Figure 3 shown, in this embodiment, Step Two includes the following specific steps:

[0056] Step S21: Collect tile adhesive data. The tile adhesive data includes the tile adhesive fluidity and obtain the weight of the floor tile;

[0057] In specific implementation, a fluidity tester is used to measure the tile adhesive that has been stirred and is ready for use, and the fluidity of the tile adhesive is obtained. To ensure the accuracy of the fluidity, it can be measured multiple times and the average value is taken; the floor tile laying robot uses a weight sensor to collect the weight of the floor tile when clamping the floor tile.

[0058] Step S22: Divide the tile adhesive fluidity and floor tile weight data sets into an 80% training data set and a 20% test data set. Input the 80% training data set into the tile adhesive overflow amount neural network analysis model for training and obtain the initial tile adhesive overflow amount neural network analysis model. Then input the 20% test data set into the initial tile adhesive overflow amount neural network analysis model for testing to obtain the tile adhesive overflow amount neural network analysis model with the highest accuracy in analyzing the tile adhesive overflow amount;

[0059] Step S23: Obtain the tile adhesive overflow amount based on the tile adhesive overflow amount neural network analysis model. The tile adhesive overflow amount neural network analysis model includes an output strategy formula for specific neurons. The output strategy formula for specific neurons is: , where represents the output of the h-th neuron in the (j + 1)-th layer of the tile adhesive overflow amount neural network analysis model, represents the connection weight between the r-th neuron in the j-th layer and the h-th neuron in the (j + 1)-th layer of the tile adhesive overflow amount neural network analysis model, represents the output of the r-th neuron in the j-th layer of the tile adhesive overflow amount neural network analysis model, represents the bias of the linear relationship between the r-th neuron in the j-th layer and the h-th neuron in the (j + 1)-th layer of the tile adhesive overflow amount neural network analysis model, represents the Sigmoid activation function;

[0060] Step S24: Calculate the influence value of the tile adhesive overflow amount on the laying flatness based on the tile adhesive overflow amount. The influence value of the tile adhesive overflow amount on the laying flatness is calculated by the following formula: , where represents the influence value of the tile adhesive overflow amount on the laying flatness, represents the tile adhesive overflow amount corresponding to the laying of the i-th floor tile, represents the safety threshold of the tile adhesive overflow amount. In specific implementation, the safety threshold of the tile adhesive overflow amount is obtained through floor tile laying experiments. Lay several floor tiles, record the tile adhesive overflow amount during laying, screen out the areas that significantly affect the flatness, count the tile adhesive overflow amount in this area and output it as a set, and select the minimum value of the tile adhesive overflow amount in this set as the safety threshold of the tile adhesive overflow amount.

[0061] Step Three: Collect the operation data of the floor tile laying robot and analyze the influence of the operation state of the floor tile laying robot on the laying flatness;

[0062] As Figure 4 shown, in this embodiment, step three includes the following specific steps:

[0063] Step S31: Collect the operation data of the floor tile laying robot. The operation data of the floor tile laying robot includes the vibration amplitude and vibration frequency;

[0064] Step S32: Calculate the influence value of the robot operation on the laying flatness based on the vibration amplitude and vibration frequency. The influence value of the robot operation on the laying flatness is calculated by the following formula: , where represents the influence value of the robot operation on the laying flatness, represents the monitoring duration of the robot operation, represents the vibration frequency of the robot operation at time t, represents the standard vibration frequency of the robot operation, represents the vibration amplitude of the robot operation at time t, represents the standard vibration amplitude of the robot operation, represents the time integral. In this embodiment, when the vibration frequency and vibration amplitude are larger, it indicates that the fault condition of the floor tile laying robot is more serious, and then the influence value of the robot operation on the laying flatness is larger. In specific implementation, the standard vibration frequency and standard vibration amplitude of the robot operation are obtained through robot operation experiments. Select several floor tile laying robots that meet the production qualification standards for operation monitoring, record the vibration frequency and vibration amplitude of the robots during operation, and take the average values of the collected vibration frequency and vibration amplitude as the standard vibration frequency and standard vibration amplitude of the robot operation respectively.

[0065] Step four: Analyze the comprehensive flatness of floor tile laying based on the influence of floor tile defects on the laying flatness, the influence of tile adhesive overflow on the laying flatness, and the influence of the operation state of the floor tile laying robot on the laying flatness;

[0066] In this embodiment, step four includes the following specific steps:

[0067] Calculate the comprehensive flatness of floor tile laying based on the influence value of floor tile defects on the laying flatness, the influence value of tile adhesive overflow on the laying flatness, and the influence value of the robot operation on the laying flatness. The comprehensive flatness of floor tile laying is calculated by the following formula: , where represents the comprehensive flatness, represents the proportion coefficient of floor tile defects, represents the proportion coefficient of tile adhesive overflow, represents the proportion coefficient of robot operation.

[0068] In specific implementation, the floor tile defect ratio coefficient, the tile adhesive overflow ratio coefficient, and the robot operation ratio coefficient are obtained through fitting software. Select several areas for floor tile paving, collect pictures of the bottom of the floor tiles, analyze the influence value of floor tile defects on paving flatness, collect tile adhesive data, analyze the influence value of tile adhesive overflow on paving flatness, collect the operation data of the floor tile paving robot, analyze the influence value of the operation state of the floor tile paving robot on paving flatness, and obtain the comprehensive flatness of the area based on the influence value of floor tile defects on paving flatness, the influence value of tile adhesive overflow on paving flatness, and the influence value of the operation state of the floor tile paving robot on paving flatness. After actual floor tile paving, obtain the comprehensive flatness of the area again, and input the two obtained comprehensive flatness values into the fitting software to output the floor tile defect ratio coefficient, the tile adhesive overflow ratio coefficient, and the robot operation ratio coefficient corresponding to the highest accuracy comprehensive flatness, which are used as the values of the floor tile defect ratio coefficient, the tile adhesive overflow ratio coefficient, and the robot operation ratio coefficient in this embodiment.

[0069] Step Five: Analyze whether the floor tiles meet the paving standard based on the comprehensive flatness.

[0070] In this embodiment, Step Five includes the following specific steps:

[0071] Analyze whether the floor tiles meet the paving standard based on the comparison result between the comprehensive flatness and the preset comprehensive flatness standard value. If the comprehensive flatness is greater than or equal to the preset comprehensive flatness standard value, output that the floor tiles meet the paving standard. If the comprehensive flatness is less than the preset comprehensive flatness standard value, output that the floor tiles do not meet the paving standard.

[0072] As Figure 5 shown, the embodiment of the present application provides a running monitoring system for a floor tile paving robot based on machine vision, which is implemented based on the above-mentioned running monitoring method for a floor tile paving robot based on machine vision, including a vision acquisition module for collecting pictures of the bottom of the floor tiles as comparison images and standard floor tile bottom images, and outputting the comparison images and the standard floor tile bottom images in the form of pixel value sets;

[0073] A floor tile defect influence analysis module for calculating the floor tile defect value based on the comparison image and the standard floor tile bottom image, and calculating the influence value of the floor tile defect on the paving flatness based on the floor tile defect value;

[0074] A data acquisition module for collecting tile adhesive data, where the tile adhesive data includes the fluidity of the tile adhesive, collecting the weight of the floor tile, and collecting the operation data of the floor tile paving robot, where the operation data of the floor tile paving robot includes the vibration amplitude and the vibration frequency;

[0075] The tile adhesive overflow impact analysis module is used to train a neural network analysis model for tile adhesive overflow amount based on the tile adhesive fluidity and floor tile weight datasets, obtain the tile adhesive overflow amount based on the neural network analysis model for tile adhesive overflow amount, and calculate the impact value of the tile adhesive overflow amount on the laying flatness;

[0076] The robot operation impact analysis module is used to calculate the impact value of the robot operation on the laying flatness based on the vibration amplitude and vibration frequency;

[0077] The comprehensive flatness analysis module is used to calculate the comprehensive flatness of the floor tile laying based on the impact value of the floor tile defect on the laying flatness, the impact value of the tile adhesive overflow amount on the laying flatness, and the impact value of the robot operation on the laying flatness;

[0078] The standard judgment module is used to analyze whether the floor tile meets the laying standard based on the comparison result between the comprehensive flatness and the preset comprehensive flatness standard value. If the comprehensive flatness is greater than or equal to the preset comprehensive flatness standard value, it outputs that the floor tile meets the laying standard. If the comprehensive flatness is less than the preset comprehensive flatness standard value, it outputs that the floor tile does not meet the laying standard.

[0079] The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. Among them, when the computer program is executed by a processor, the above-mentioned method for monitoring the operation of a floor tile laying robot based on machine vision is implemented.

[0080] Computer-readable media include both permanent and non-permanent, removable and non-removable media and can be implemented by any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0081] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this application can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0082] The above has introduced the technical content provided by this application in detail. Specific examples are used in this application to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A floor tile laying robot operation monitoring method based on machine vision, characterized in that: The specific steps include: Step 1: Collect pictures of the bottom of the floor tiles and analyze the impact of floor tile defects on paving flatness; Step 2: Collect tile adhesive data and analyze the impact of tile adhesive overflow on paving flatness; Step 3: Collecting the operation data of the floor tile paving robot, wherein the operation data of the floor tile paving robot includes vibration amplitude and vibration frequency, and analyzing the influence of the operation state of the floor tile paving robot on the paving flatness; Step 4: Analyze the comprehensive flatness of floor tile paving based on the influence of floor tile defects on paving flatness, the influence of tile adhesive overflow on paving flatness, and the influence of the operation state of the floor tile paving robot on paving flatness; Step 5: Analyze whether the floor tiles meet the paving standards based on comprehensive flatness; The step five includes the following specific steps: Based on the comparison results between the comprehensive flatness and the preset comprehensive flatness standard value, it is analyzed whether the floor tiles meet the paving standards. If the comprehensive flatness is greater than or equal to the preset comprehensive flatness standard value, the output floor tiles meet the paving standards. If the comprehensive flatness is less than the preset comprehensive flatness standard value, the output floor tiles do not meet the paving standards.

2. The machine vision-based floor tile paving robot operation monitoring method according to claim 1, characterized in that: The step 1 includes the following specific steps: Step S11, collecting a floor tile bottom image and setting it as a comparison image, obtaining a pre-collected standard floor tile bottom image, and outputting the comparison image and the standard floor tile bottom image in the form of a pixel value set; Step S12: Calculate the floor tile defect value based on the comparison image and the standard floor tile bottom image, and the floor tile defect value is calculated by the following formula: , where represents the tile defect value of the i-th tile, represents the pixel value set of the contrast image corresponding to the i-th floor tile, A set of pixel values ​​representing the bottom image of a standard floor tile. represents the intersection, represents a union; Step S13: Calculate the influence of the floor tile defect on the paving flatness based on the floor tile defect value. The influence of the floor tile defect on the paving flatness is calculated by the following formula: , where Indicates the impact of floor tile defects on paving flatness. Indicates the number of floor tiles.

3. The machine vision-based floor tile paving robot operation monitoring method according to claim 2, characterized in that: The step 2 includes the following specific steps: Step S21, collecting tile adhesive data, wherein the tile adhesive data includes the fluidity of the tile adhesive, and obtaining the weight of the floor tiles; Step S22, dividing the tile adhesive fluidity and floor tile weight data set into 80% training data set and 20% test data set, inputting 80% of the training data set into the tile adhesive overflow neural network analysis model for training and obtaining an initial tile adhesive overflow neural network analysis model, and then inputting 20% ​​of the test data set into the initial tile adhesive overflow neural network analysis model for testing, to obtain a tile adhesive overflow neural network analysis model with the highest tile adhesive overflow analysis accuracy; Step S23, obtaining the tile glue overflow amount based on the tile glue overflow amount neural network analysis model, the tile glue overflow amount neural network analysis model includes an output strategy formula of a specific neuron, and the output strategy formula of the specific neuron is: , where represents the output of the h-th neuron in the j+1th layer of the neural network analysis model for tile adhesive overflow, represents the connection weight of the j-th layer neuron r and the j+1-th layer h-th neuron in the neural network analysis model of tile adhesive overflow, represents the output of the j-th layer neuron r of the neural network analysis model of tile adhesive overflow, The offset of the linear relationship between the j-th layer neuron r and the j+1-th layer h-term neuron of the neural network analysis model for tile adhesive overflow, Represents the Sigmoid activation function; Step S24, calculating the influence value of the tile adhesive overflow on the paving flatness based on the tile adhesive overflow, wherein the influence value of the tile adhesive overflow on the paving flatness is calculated by the following formula: , where Indicates the influence of tile adhesive overflow on paving flatness. represents the amount of tile adhesive overflow corresponding to the ith floor tile after paving, Indicates the safe threshold for tile adhesive overflow.

4. The machine vision-based floor tile paving robot operation monitoring method according to claim 3, characterized in that: The step three includes the following specific steps: Step S31, collecting operation data of the floor tile paving robot, wherein the operation data of the floor tile paving robot includes vibration amplitude and vibration frequency; Step S32: Calculate the impact of the robot operation on the paving flatness based on the vibration amplitude and vibration frequency. The impact of the robot operation on the paving flatness is calculated by the following formula: , where Indicates the impact of robot operation on paving flatness. Indicates the robot operation monitoring time. represents the vibration frequency of the robot at time t, Indicates the standard vibration frequency of the robot operation. represents the vibration amplitude of the robot at time t, Indicates the standard vibration amplitude of the robot operation. represents the time integral.

5. The machine vision-based floor tile paving robot operation monitoring method as claimed in claim 4, characterized in that: The step 4 includes the following specific steps: The comprehensive flatness of the floor tile paving is calculated based on the influence of floor tile defects on the paving flatness, the influence of tile adhesive overflow on the paving flatness, and the influence of robot operation on the paving flatness. The comprehensive flatness of the floor tile paving is calculated by the following formula: , where Indicates the overall flatness, represents the floor tile defect ratio, represents the coefficient of tile adhesive overflow, Indicates the robot operation ratio.

6. A floor tile laying robot operation monitoring system based on machine vision, used to implement the floor tile laying robot operation monitoring method based on machine vision as claimed in any one of claims 1 to 5, characterized in that: It includes a visual acquisition module, which is used to collect the bottom image of the floor tile and set it as a comparison image and a standard bottom image of the floor tile, and output the comparison image and the standard bottom image of the floor tile in the form of a pixel value set; A floor tile defect impact analysis module is used to calculate the floor tile defect value based on the comparison image and the standard floor tile bottom image, and calculate the impact value of the floor tile defect on the paving flatness based on the floor tile defect value; A data acquisition module is used to collect tile adhesive data, including tile adhesive fluidity, tile weight, and tile laying robot operation data, including vibration amplitude and vibration frequency; The tile adhesive overflow impact analysis module is used to train a tile adhesive overflow neural network analysis model based on the tile adhesive fluidity and floor tile weight data set, obtain the tile adhesive overflow based on the tile adhesive overflow neural network analysis model, and calculate the impact value of the tile adhesive overflow on the paving flatness based on the tile adhesive overflow; The robot operation impact analysis module is used to calculate the impact of the robot operation on the paving flatness based on the vibration amplitude and vibration frequency; Comprehensive flatness analysis module, used to calculate the comprehensive flatness of floor tile paving based on the influence of floor tile defects on paving flatness, the influence of tile adhesive overflow on paving flatness, and the influence of robot operation on paving flatness; The standard judgment module is used to analyze whether the floor tiles meet the paving standards based on the comparison results between the comprehensive flatness and the preset comprehensive flatness standard value. If the comprehensive flatness is greater than or equal to the preset comprehensive flatness standard value, the output is that the floor tiles meet the paving standards. If the comprehensive flatness is less than the preset comprehensive flatness standard value, the output is that the floor tiles do not meet the paving standards.

7. A computer-readable storage medium, characterized in that: It includes computer operating instructions. When the computer operating instructions are executed on a computer, the computer executes the machine vision-based floor tile paving robot operation monitoring method as described in any one of claims 1 to 5.

Images