A kind of straightness measurement method and equipment of fully mechanized working face scraper

Abstract

The application provides a method and equipment for measuring straightness of a scraper of a fully-mechanized coal mining face. The method comprises the following steps: collecting planar target images arranged at specified positions of the scraper body according to predetermined rules; determining a rotation matrix and a translation matrix of each planar target image in a corresponding camera coordinate system; calculating three-dimensional coordinates of all planar target center points in the corresponding camera coordinate system; mapping the three-dimensional coordinates of all the planar target center points to a global coordinate system; and calculating straightness in three-axis directions of the global coordinate system based on the three-dimensional coordinates of all the planar target center points in the global coordinate system, fitting the overall spatial form of the scraper, and realizing measurement of the straightness of the scraper.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual measurement, and in particular to a method, device and equipment for measuring straightness of a scraper conveyor at a fully mechanized coal mining face, and a storage medium. BACKGROUND

[0002] Coal mining equipment at a fully mechanized coal mining face mainly includes a coal mining machine, a scraper conveyor and hydraulic supports. According to the Coal Mine Safety Regulations, the fully mechanized coal mining face must ensure "three straight and two flat" during mining, i.e. the coal wall, the scraper conveyor and the hydraulic supports must be straight, and the roof and floor must be flat. Among the three machines at the working face, the scraper conveyor not only serves as a track for the coal mining machine, but also as a moving point for the hydraulic supports during shifting. Therefore, ensuring the straightness of the scraper conveyor is the key to achieving "three straight and two flat" at the working face.

[0003] Traditional methods for detecting straightness of the scraper conveyor mainly include the following: the first method is to directly measure the straightness of the scraper conveyor by taking the scraper conveyor itself as a reference, such as by arranging three-axis accelerometers and gyroscopes in the middle trough of each scraper conveyor, using integral operation to obtain the three-axis angles and displacements of each middle trough, and then using curve fitting to obtain the straightness of the entire scraper conveyor. This method is simple and direct, but requires a large number of sensors, which increases the workload of installation and maintenance on site, and is prone to cumulative errors. The second method is to take the hydraulic supports as a reference, measure the distance between each support and the scraper conveyor by using displacement sensors, and then determine the straightness of the scraper conveyor according to the relative displacement between adjacent supports and the scraper conveyor. This method does not take into account the up-and-down floating between the supports and the scraper conveyor, which can lead to large errors in the measurement results. The third method is to take the coal mining machine as a reference, use the strapdown inertial navigation system on the coal mining machine to obtain its own running track, and then obtain the position of each section of the scraper conveyor by inversion to realize the straightness measurement of the scraper conveyor. This is the most commonly used method, but the cumulative error problem of the inertial navigation system has been difficult to solve.

[0004] In recent years, with the development of intelligent coal mines, the working face environment and illumination conditions have been greatly improved, and higher requirements for high-precision measurement of the pose parameters of fully mechanized equipment have been put forward. To this end, a patent with application number 202110264891.1 provides a method and device for detecting straightness of a scraper conveyor based on video recognition, which identifies and instance segments the scraper conveyor through image recognition, extracts the outer contour coordinate values of the scraper conveyor using an algorithm, and calculates the linear correlation coefficient to determine whether the straightness of the scraper conveyor meets the requirements. This method can realize non-contact measurement of the straightness of the scraper conveyor, but can only detect the straightness of the scraper conveyor within the field of view of a single camera, and is a single-dimensional measurement, which cannot realize three-dimensional straightness measurement of the scraper conveyor in the entire scene of the working face. SUMMARY

[0005] The application provides a kind of full-mechanized coal mining face scraper straightness measurement method, device, equipment, storage medium, to realize the panoramic straightness detection of scraper;And realize the straightness detection of three-dimensional direction.

[0006] To this end, the first object of the application is to provide a kind of full-mechanized coal mining face scraper straightness measurement method, comprising:

[0007] Collect the planar target image arranged at the specified position of the scraper body and according to the predetermined rule, determine the rotation matrix and translation matrix of each planar target image in the corresponding camera coordinate system;

[0008] Based on the rotation matrix and translation matrix, calculate the three-dimensional coordinates of all planar target center points in each planar target image in the corresponding camera coordinate system;

[0009] Map the three-dimensional coordinates of all planar target center points to the global coordinate system, and based on the three-dimensional coordinates of all planar target center points in the global coordinate system, calculate the straightness in the three-axis direction of the global coordinate system, fit the overall spatial form of the scraper, and realize the measurement of the straightness of the scraper.

[0010] Among them, when arranging the planar target image at the specified position of the scraper body and according to the predetermined rule, it includes:

[0011] Select the hydraulic support at the end of the working face as the starting point, set the interval between the hydraulic supports, and install wide-angle cameras on the top beams of the hydraulic supports with a preset interval;When installing, the lens field of view is downward, ensuring that the fields of view of all cameras are connected in sequence to form a total field of view, and the scraper is completely present in the total field of view of the cameras;

[0012] Cyclically arrange three planar targets with different visual features at the cable groove outside the scraper body, so that the shooting field of view of each camera completely captures the three planar targets, and the two planar targets on both sides are in the overlapping area of the fields of view of adjacent cameras.

[0013] Among them, in the step of determining the rotation matrix and translation matrix of each planar target image in the corresponding camera coordinate system, it includes:

[0014] Use Zhang Zhengyou calibration method to calibrate all cameras respectively, and obtain the intrinsic matrix and distortion matrix of the corresponding camera;

[0015] Based on the calibration results of the cameras, correct the distortion of the planar target images collected by the corresponding cameras;

[0016] Identify the three planar targets in each planar target image after distortion correction, and calculate the rotation matrix and translation matrix of the three planar targets in the corresponding camera coordinate system.

[0017] In the step of calculating the three-dimensional coordinates of all the center points of the planar targets in each planar target image in the corresponding camera coordinate system based on the rotation matrix and the translation matrix, the three-dimensional coordinates of the center points of the planar targets in the corresponding camera coordinate system are calculated based on the rotation matrix and the translation matrix. The calculation formula is represented as:

[0018]

[0019] wherein, is the pixel coordinate corresponding to the center point P of the jth planar target in the field of view of the ith camera, K i is the intrinsic matrix of the ith camera; respectively represent the rotation matrix and the translation matrix corresponding to the jth planar target in the camera coordinate system of the ith camera; is the three-dimensional coordinate corresponding to the center point P of the jth planar target in the camera coordinate system of the ith camera,

[0020] In the step of mapping the three-dimensional coordinates of all the center points of the planar targets to the global coordinate system, the following steps are included:

[0021] The coordinate system of the first camera among all the cameras is defined as the global coordinate system, and the global coordinates of the three planar targets in the field of view of the first camera are determined.

[0022] The planar targets in the overlapping region of the field of view of the first camera and the second camera are taken as the pose transfer medium, the first transformation matrix of the second camera to the first camera is calculated, and the coordinates of the remaining two center points of the planar targets in the field of view of the second camera are converted into global coordinates through the first transformation matrix.

[0023] The planar targets in the overlapping region of the field of view of the second camera and the third camera are taken as the pose transfer medium, the second transformation matrix of the third camera to the second camera is calculated, and the third transformation matrix of the third camera to the first camera is calculated by using the first transformation matrix and the second transformation matrix; the coordinates of the remaining two center points of the planar targets in the field of view of the third camera are converted into global coordinates through the third transformation matrix.

[0024] In this way, the global coordinates of the three center points of the planar targets in the field of view of all the cameras in the global coordinate system are obtained.

[0025] In the step of calculating the straightness in the three-axis direction of the global coordinate system based on the three-dimensional coordinates of all the center points of the planar targets in the global coordinate system, the straightness calculation formula is used to calculate the straightness of the scraper in the X, Y and Z directions of the global coordinate system, respectively.

[0026] Further comprising a step of three-dimensionally visualizing the spatial form of the scraper conveyor; visualizing the pose of each planar target in the global coordinate system, and fitting the overall spatial form of the scraper conveyor according to the same, to realize the detection of the straightness of the scraper conveyor.

[0027] The second object of the present application is to provide a scraper straightness measurement device for fully-mechanized coal mining face, comprising:

[0028] The acquisition module is configured to acquire planar target images arranged at designated positions of the scraper body and arranged according to a predetermined rule, and determine a rotation matrix and a translation matrix of each planar target image in a corresponding camera coordinate system;

[0029] The calculation module is configured to calculate three-dimensional coordinates of all planar target center points in each planar target image in the corresponding camera coordinate system based on the rotation matrix and the translation matrix;

[0030] The measurement module is configured to map the three-dimensional coordinates of all planar target center points to a global coordinate system, and calculate straightness in three-axis directions of the global coordinate system based on the three-dimensional coordinates of all planar target center points in the global coordinate system, fit the overall spatial form of the scraper, and realize the measurement of the straightness of the scraper.

[0031] The third object of the present application is to provide an electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform each step in the method of the foregoing technical solution.

[0032] The fourth object of the present application is to provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to perform each step in the method according to the foregoing technical solution.

[0033] Different from the prior art, the scraper straightness measurement method for fully-mechanized coal mining face provided by the present application acquires planar target images arranged at designated positions of the scraper body and arranged according to a predetermined rule, determines a rotation matrix and a translation matrix of each planar target image in a corresponding camera coordinate system, calculates three-dimensional coordinates of all planar target center points in each planar target image in the corresponding camera coordinate system, maps the three-dimensional coordinates of all planar target center points to a global coordinate system, calculates straightness in three-axis directions of the global coordinate system based on the three-dimensional coordinates of all planar target center points in the global coordinate system, fits the overall spatial form of the scraper, and realizes the measurement of the straightness of the scraper. Through the present application, panoramic straightness detection of the scraper can be realized, and straightness detection in three-dimensional directions can be realized. Attached Figure Description

[0034] The present invention and / or its additional aspects and advantages will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0035] Figure 1 This is a flowchart illustrating a method for measuring the straightness of a scraper conveyor in a fully mechanized mining face, provided by the present invention.

[0036] Figure 2 This is a schematic diagram of the arrangement of the planar target and the camera in a method for measuring the straightness of a scraper conveyor in a fully mechanized mining face provided by the present invention.

[0037] Figure 3 This is a schematic diagram of the structure of a straightness measuring device for a scraper conveyor in a fully mechanized mining face provided by the present invention.

[0038] Figure 4 This is a schematic diagram of the structure of a non-transitory computer-readable storage medium provided by the present invention. Detailed Implementation

[0039] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0040] like Figure 1 As shown, an embodiment of the present invention provides a method for measuring the straightness of a scraper conveyor in a fully mechanized mining face, comprising:

[0041] S110: Acquire planar target images set at a designated position on the scraper conveyor body and arranged according to a predetermined rule, and determine the rotation matrix and translation matrix of each planar target image in the corresponding camera coordinate system.

[0042] When setting and arranging planar target images at designated positions on the scraper conveyor body according to predetermined rules, the following is included:

[0043] Select the hydraulic support at the end of the working face as the starting point, set the interval of the hydraulic support, and install a wide-angle shooting camera on the top beam of the hydraulic support with a preset interval; when installing, the lens field of view is downward, ensuring that the fields of view of all shooting cameras are connected in sequence to form the total field of view, and the scraper machine is completely in the total field of view of the shooting cameras.

[0044] Three kinds of planar targets with different visual features are cyclically arranged at the cable groove outside the machine body of the scraper, so that the visual field of each shooting camera can completely capture the three kinds of planar targets, and the two kinds of planar targets on the two sides are in the overlapping area of the visual field of the adjacent shooting cameras. The arrangement of the planar targets and the shooting cameras is as shown in Figure 2 .

[0045] Specifically, a wide-angle shooting camera is installed every n hydraulic support, and the value of n is determined according to the actual site conditions, so that there is a maximum value of a certain range of visual field overlap between adjacent shooting cameras; and the same is true until the camera arrangement of the entire working face scene is completed. After the installation is completed, the shooting cameras are labeled 1, 2, 3…N.

[0046] After the shooting cameras are installed, three kinds of planar targets with different visual features are cyclically arranged at the cable groove outside the machine body of the scraper. In this embodiment, three kinds of ArUco targets are taken as an example, and are numbered 1, 2 and 3. The arrangement principle of the three kinds of ArUco targets is to ensure that each shooting camera can completely capture the three kinds of planar targets in the visual field, wherein the 1st and 3rd targets are respectively located in the overlapping area of the visual field of the current shooting camera and the adjacent two shooting cameras, and the 2nd target is located in the central area of the visual field of the current shooting camera, as shown in Figure 2 . In the figure, 1 represents a hydraulic support, 2 represents a planar target, 3 represents a scraper, the planar targets 2 are sequentially arranged along the cable groove of the scraper, and 4 represents a shooting camera. As shown in Figure 2 , the 1st shooting camera and the 2nd shooting camera are arranged with the 3rd planar target in the overlapping area of the visual field, and the 2nd shooting camera and the 3rd shooting camera are arranged with the 1st planar target in the overlapping area of the visual field.

[0047] In the step of determining the rotation matrix and the translation matrix of each planar target image in the corresponding shooting camera coordinate system, the following steps are included:

[0048] The camera calibration is performed on all the shooting cameras respectively by using the Zhang Zhengyou calibration method, so as to obtain the intrinsic matrix and the distortion matrix of the corresponding shooting camera.

[0049] Based on the calibration result of the shooting camera, the planar target images collected by the corresponding shooting camera are corrected for distortion.

[0050] The three kinds of planar targets in each planar target image after the distortion correction are identified, and the rotation matrix and the translation matrix of the three kinds of planar targets in the corresponding shooting camera coordinate system are calculated respectively.

[0051] Specifically, the camera calibration is performed on the N wide-angle shooting cameras respectively by using the Zhang Zhengyou calibration method, so as to obtain the intrinsic matrix K and the distortion matrix ξ(k1, k2, k3, k4, k5, k6, p1, p2) corresponding to each shooting camera.

[0052] wherein the intrinsic matrix wherein: f x , f y is the normalized focal length of the camera; u0, v0 is the pixel coordinate of the image center point.

[0053] The distortion matrix ξ: k1, k2, k3, k4, k5, k6 represents the radial distortion coefficient of the camera, and p1, p2 represents the tangential distortion coefficient of the camera.

[0054] Each shooting camera respectively acquires images of the planar targets and the current scraper section within its field of view, and according to the shooting camera calibration result, the acquired images are corrected for distortion to obtain corrected images.

[0055] The three ArUco targets in the corrected images are recognized and detected, the salient corner points and identity codes of each ArUco target are obtained, and the rotation matrix and the translation matrix

[0056] S120: Based on the rotation matrix and the translation matrix, the three-dimensional coordinates of all planar target center points in each planar target image under the corresponding shooting camera coordinate system are calculated.

[0057] According to the rotation matrix and the translation matrix, the three-dimensional coordinates of the center points P of the three ArUco targets in the image under the shooting camera coordinate system are calculated The specific calculation formula is as follows:

[0058]

[0059] wherein: is the pixel coordinate corresponding to the center point P of the jth planar target in the field of view of the ith shooting camera, K i is the intrinsic matrix of the ith shooting camera; respectively represent the rotation matrix and the translation matrix corresponding to the jth planar target under the camera coordinate system of the ith shooting camera; is the three-dimensional coordinate corresponding to the center point P of the jth planar target under the camera coordinate system of the ith shooting camera,

[0060] S130: Map the three-dimensional coordinates of all planar target center points to the global coordinate system, and based on the three-dimensional coordinates of all planar target center points under the global coordinate system, calculate the straightness in the three-axis direction of the global coordinate system, fit the overall spatial form of the scraper, and realize the measurement of the straightness of the scraper.

[0061] In the step of mapping the three-dimensional coordinates of all planar target center points to a global coordinate system, the following steps are included:

[0062] The coordinate system of the first camera in all cameras is defined as the global coordinate system, and the global coordinates of the three planar targets in the field of view of the first camera are determined.

[0063] The planar target in the overlapping area of the field of view of the first camera and the second camera is used as the pose transfer medium, the first transformation matrix of the second camera to the first camera is calculated, and the coordinates of the remaining two planar target center points in the field of view of the second camera are converted to global coordinates through the first transformation matrix.

[0064] The planar target in the overlapping area of the field of view of the second camera and the third camera is used as the pose transfer medium, the second transformation matrix of the third camera to the second camera is calculated, and the third transformation matrix of the third camera to the first camera is calculated using the first transformation matrix and the second transformation matrix. The coordinates of the remaining two planar target center points in the field of view of the third camera are converted to global coordinates through the third transformation matrix.

[0065] Similarly, the global coordinates of the three planar target center points in the field of view of all cameras in the global coordinate system are obtained.

[0066] Specifically, the camera coordinate system of the first camera is defined as the global coordinate system O-XYZ (the purpose is to convert all planar target coordinates to this coordinate system to achieve coordinate unification), and the global coordinates of the three ArUco target center points in the field of view of the first camera are respectively

[0067] The planar target in the overlapping area of the field of view of the first camera and the second camera is used as the pose transfer medium, and the transformation matrix of the second camera to the first camera is calculated Then, the transformation matrix is The coordinates of the remaining two planar targets (i.e., 2 and 1) in the field of view of the second camera are converted to the global coordinate system O-XYZ, and the coordinates of the three ArUco target center points in the field of view of the second camera are converted to the global coordinate system O-XYZ. The global coordinates corresponding to the three ArUco target center points in the field of view of the second camera are respectively

[0068] The planar target in the overlapping area of the field of view of the second camera and the third camera is used as the pose transfer medium, and the transformation matrix of the third camera to the second camera is calculated Further, the transformation matrix from the third camera to the first camera is obtained Then the transformation matrix is obtained The center point coordinates of the remaining two planar targets (i.e., No. 2 and No. 3) in the field of view of the No. 3 camera are converted into the global coordinate system O-XYZ, and the coordinates of the center points of the No. 2 and No. 3 targets in the global coordinate system are obtained as follows:

[0069] By analogy, the coordinates of the center points of the three planar targets in the fields of view of the N cameras are sequentially converted into the global coordinate system O-XYZ, and finally the global coordinate set of the center points of all the 3N ArUco targets is obtained as follows:

[0070] Based on the three-dimensional coordinates of all the planar target center points in the global coordinate system, the straightness in the three-axis directions of the global coordinate system is calculated, and the straightness of the scraper in the X, Y and Z directions of the global coordinate system is calculated respectively using the straightness calculation formula.

[0071] According to each coordinate value, the straightness of the scraper conveyor in the X, Y and Z directions is calculated respectively; the pose of each planar target in the global coordinate system is visually displayed, and the overall spatial form of the scraper conveyor is fitted according to this, so as to realize the detection of the straightness of the scraper conveyor.

[0072] As shown in Figure 3 The present application also provides a fully-mechanized coal mining face scraper straightness measuring device, which comprises:

[0073] The acquisition module 310 is configured to acquire planar target images arranged at specified positions of the scraper body and arranged according to a predetermined rule, and determine a rotation matrix and a translation matrix of each planar target image in a corresponding camera coordinate system;

[0074] The calculation module 320 is configured to calculate three-dimensional coordinates of all the planar target center points in the corresponding camera coordinate system based on the rotation matrix and the translation matrix;

[0075] The measurement module 330 is configured to map the three-dimensional coordinates of all the planar target center points to a global coordinate system, calculate the straightness in the three-axis directions of the global coordinate system based on the three-dimensional coordinates of all the planar target center points in the global coordinate system, and fit the overall spatial form of the scraper, so as to realize the measurement of the straightness of the scraper;

[0076] The display module 340 is configured to perform three-dimensional visual display on the spatial form of the scraper; visually display the pose of each planar target in the global coordinate system, and fit the overall spatial form of the scraper conveyor according to this, so as to realize the detection of the straightness of the scraper conveyor.

[0077] To achieve the embodiments, the application further provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform each step in the method for measuring straightness of a scraper conveyor in a fully-mechanized coal mining face.

[0078] As shown in Figure 4 The non-transitory computer readable storage medium includes a memory 810 storing instructions executable by a processor 820 for measuring straightness of a scraper conveyor in a fully-mechanized coal mining face. Optionally, the storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0079] To achieve the embodiments, the application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method for measuring straightness of a scraper conveyor in a fully-mechanized coal mining face according to the embodiments of the application.

[0080] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the different embodiments or examples described in the present specification and the features of the different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0081] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0082] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0083] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0084] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the described embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0085] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the method of the embodiment can be instructed by a program to complete the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0086] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0087] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the embodiments within the scope of the present application.

Claims

1. A method for measuring straightness of a scraper conveyor at a fully-mechanized coal mining face, characterized in that, The method comprises the following steps: Collecting planar target images arranged at designated positions of the machine body of the scraper conveyor and arranged according to predetermined rules, and determining a rotation matrix and a translation matrix of each planar target image in the corresponding camera coordinate system; Based on the rotation matrix and the translation matrix, calculating the three-dimensional coordinates of all planar target center points in each planar target image in the corresponding camera coordinate system; Mapping the three-dimensional coordinates of all planar target center points to the global coordinate system, and based on the three-dimensional coordinates of all planar target center points in the global coordinate system, calculating the straightness in the three-axis direction of the global coordinate system, fitting the overall spatial form of the scraper conveyor, and realizing the measurement of the straightness of the scraper conveyor; When arranging the planar target images at designated positions of the machine body of the scraper conveyor and according to predetermined rules, the method comprises the following steps: Selecting a hydraulic support at the end of the working face as the starting point, setting the interval of the hydraulic supports, and installing wide-angle cameras on the top beams of the hydraulic supports at the predetermined interval; when installing, the lens field of view is downward, ensuring that the fields of view of all the cameras are sequentially connected to form a total field of view, and the scraper conveyor is completely present in the total field of view of the cameras; Cyclically arranging three planar targets with different visual features at the cable groove outside the machine body of the scraper conveyor, so that the shooting field of view of each camera completely captures the three planar targets, and the two planar targets on the two sides are in the field of view overlap area of the adjacent cameras, wherein the three planar targets with different visual features arranged cyclically at the cable groove outside the machine body of the scraper conveyor comprise three ArUco targets numbered 1, 2 and 3, wherein the No. 1 and No. 3 targets are respectively located in the field of view overlap area of the current camera and the adjacent two side cameras, and the No. 2 target is located in the central field of view of the current camera; Further comprising the following steps: three-dimensional visualizing the spatial form of the scraper conveyor; visualizing the pose of each planar target in the global coordinate system, and fitting the overall spatial form of the scraper conveyor according to the visualized pose, thereby realizing the detection of the straightness of the scraper conveyor.

2. The method according to claim 1, characterized in that, In the step of determining the rotation matrix and the translation matrix of each planar target image in the corresponding camera coordinate system, the method comprises the following steps: Using Zhang Zhengyou calibration method to calibrate all the cameras respectively, and obtaining the intrinsic matrix and the distortion matrix of the corresponding camera; Based on the calibration results of the cameras, correcting the distortion of the planar target images collected by the corresponding cameras; Identifying the three planar targets in each planar target image after distortion correction, and calculating the rotation matrix and the translation matrix of the three planar targets in the corresponding camera coordinate system respectively.

3. The method according to claim 2, characterized in that, In the step of calculating the three-dimensional coordinates of all planar target center points in each planar target image in the corresponding camera coordinate system based on the rotation matrix and the translation matrix, the three-dimensional coordinates of the planar target center points in the camera coordinate system are calculated by the following formula: (i = 1, 2, …, N), and the calculation formula is represented as: wherein, Pi,j is a pixel coordinate corresponding to the center point P of the jth planar target in the field of view of the ith camera, K i is an intrinsic matrix of the ith camera; respectively represent a rotation matrix and a translation matrix corresponding to the jth planar target in the camera coordinate system of the ith camera; Pi,j is a three-dimensional coordinate corresponding to the center point P of the jth planar target in the camera coordinate system of the ith camera, 4. The method according to claim 3, characterized in that, In the step of mapping the three-dimensional coordinates of all planar target center points to the global coordinate system, the method comprises the following steps: Defining the coordinate system of the first camera as the global coordinate system, and determining the global coordinates of the three planar targets in the field of view of the first camera; Taking the planar targets in the field of view overlap area of the first camera and the second camera as the pose transmission medium, calculating the first transformation matrix from the second camera to the first camera, and converting the coordinates of the remaining two planar target center points in the field of view of the second camera into global coordinates through the first transformation matrix; The second camera and the third camera are used to capture the image of the plane target in the overlapping area of the field of view, and the second transformation matrix of the third camera to the second camera is calculated by taking the plane target in the overlapping area of the field of view of the second camera and the third camera as a pose transfer medium; the third transformation matrix of the third camera to the first camera is calculated by using the first transformation matrix and the second transformation matrix; and the center point coordinates of the remaining two plane targets in the field of view of the third camera are converted into global coordinates by the third transformation matrix. Similarly, the global coordinates of the center points of all the plane targets in the field of view of all the cameras in the global coordinate system are obtained.

5. The method according to claim 4, characterized in that, Based on the three-dimensional coordinates of all the plane target center points in the global coordinate system, the straightness of the scraper in the X, Y and Z directions of the global coordinate system is calculated by using the straightness calculation formula.

6. A device for measuring straightness of a scraper conveyor of a fully mechanized coal mining face, characterized in that, The device comprises: a collection module configured to collect images of plane targets arranged at specified positions of a body of the scraper and arranged according to a predetermined rule, and determine a rotation matrix and a translation matrix of each plane target image in a corresponding camera coordinate system; a calculation module configured to calculate three-dimensional coordinates of all the plane target center points in the corresponding camera coordinate system based on the rotation matrix and the translation matrix; a measurement module configured to map the three-dimensional coordinates of all the plane target center points to a global coordinate system, calculate straightness in three-axis directions of the global coordinate system based on the three-dimensional coordinates of all the plane target center points in the global coordinate system, fit the overall spatial form of the scraper, and realize measurement of the straightness of the scraper. When the plane target images are arranged at specified positions of the body of the scraper and arranged according to a predetermined rule, the device comprises: selecting a hydraulic support at the end of a working face as a starting point, setting a hydraulic support interval, and installing a wide-angle camera on a roof beam of a hydraulic support at a predetermined interval; when the camera is installed, the lens field of view is downward, ensuring that all the camera fields of view are sequentially connected to form a total field of view, and the scraper is completely present in the total field of view of the camera; cyclically arranging three plane targets with different visual features at the cable groove outside the body of the scraper, so that the camera field of view of each camera completely captures the three plane targets, and the two plane targets on the two sides are in the field of view overlapping area of adjacent cameras, wherein the three plane targets with different visual features arranged cyclically at the cable groove outside the body of the scraper comprise three ArUco targets numbered 1, 2 and 3, wherein the 1st and 3rd targets are respectively located in the field of view overlapping area of the current camera and the adjacent two side cameras, and the 2nd target is located in the central field of view of the current camera; The device is also used for the steps of three-dimensionally visualizing the spatial form of the scraper and visualizing the pose of each plane target in the global coordinate system, and fitting the overall spatial form of the scraper conveyor to realize detection of the straightness of the scraper conveyor.

7. An electronic device, comprising: at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method of any one of claims 1-5.

8. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing the computer to perform the steps of the method of any one of claims 1-5.

Images