A method, device and equipment for generating a rope panorama image and a storage medium

Abstract

The application discloses a rope panoramic image generation method and device, equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a multi-directional image of a to-be-detected rope, segmenting the multi-directional image to obtain rope images in the multi-directional image, correcting the coordinates of each pixel in each rope image according to a pre-fitted coordinate transformation relationship in the vertical direction and the horizontal direction to obtain the corrected coordinates of each pixel in the corresponding corrected rope image, determining the pixel values of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image, and displaying a fused rope panoramic image according to the pixel values of each pixel in the overlapping area of each corrected rope image and the pixel values of the corrected coordinates of each pixel in the non-overlapping area. The method can facilitate rope detection by detection personnel and improve detection efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating a panoramic image of a rope. Background Art

[0002] Rope health is used to characterize the safety and reliability of a rope. In industries such as industry, construction, and transportation, ropes serve as load-bearing and connection equipment, fulfilling crucial functions. Therefore, rope health testing is crucial. Typically, rope health can be analyzed by inspecting both the inside and outside of the rope.

[0003] At present, manual inspection methods such as visual inspection are generally used in the process of inspecting the outside of the rope. This inspection method requires the inspector to walk around the rope for observation. It is not only manpower-consuming and inefficient, but also prone to false alarms or missed alarms due to inadequate observation.

[0004] Therefore, the detection efficiency of the above-mentioned detection scheme for the health status of the rope is low. Summary of the Invention

[0005] The present application provides a method, device, equipment and storage medium for generating a panoramic image of a rope, which can facilitate inspection personnel to inspect the rope and improve inspection efficiency.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a method for generating a panoramic image of a rope, comprising:

[0008] Acquire multi-directional images of the rope to be tested;

[0009] Segmenting the multi-directional image to obtain a rope image in the multi-directional image;

[0010] Correcting the coordinates of each pixel in each rope image according to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship to obtain the corrected coordinates of each pixel in the corresponding corrected rope image;

[0011] determining pixel values ​​of the corrected coordinates of the pixels in the corresponding corrected rope images according to the coordinates of the pixels in the rope images;

[0012] Determine whether t is less than a direction number threshold, obtaining a first determination result, wherein the direction number threshold is greater than or equal to 3;

[0013] If the first judgment result indicates that t is less than the direction number threshold, then determining the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the tth direction based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction; wherein t is a positive integer;

[0014] If the first judgment result indicates that t is not less than the direction number threshold, the fused rope panoramic image is displayed according to the pixel value of each pixel in the overlapping area of ​​each corrected rope image and the pixel value of the corrected coordinate of each pixel in the non-overlapping area.

[0015] In some possible implementations, the horizontal coordinate transformation relationship is represented by the following formula:

[0016]

[0017] Where x″ i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the first coefficient of the pre-fitting, b1 is the second coefficient of the pre-fitting, c1 is the third coefficient of the pre-fitting, and x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.

[0018] In some possible implementations, the coordinate transformation relationship in the vertical direction is represented by the following formula:

[0019]

[0020] Among them, y i is the ordinate of the i-th pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, Y0 is the extreme value corresponding to the extreme point X0; a0 is the pre-fitted fourth coefficient, b0 is the pre-fitted fifth coefficient, c0 is the pre-fitted sixth coefficient; x i is the horizontal coordinate of the i-th pixel before correction, y i is the vertical coordinate of the i-th pixel before correction.

[0021] In some possible implementations, determining, based on the coordinates of each pixel in each rope image, the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image includes:

[0022] q(x″i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl

[0023] Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the upper right adjacent pixel of the i-th pixel before correction, q tl is the pixel value of the upper left adjacent pixel of the i-th pixel before correction, q bl is the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the vertical coordinate of the i-th pixel before correction.

[0024] In some possible implementations, the method further includes:

[0025] receiving a marking result of the rope panoramic image from a user;

[0026] If the marking result indicates that the health status score of the rope to be tested is lower than a score threshold, the corresponding position of the rope panoramic image in the rope to be tested is recorded.

[0027] In some possible implementations, the acquiring of multi-directional images of the rope to be tested includes:

[0028] At every preset distance, multi-directional images of the rope to be tested are acquired.

[0029] In a second aspect, the present application provides a device for generating a panoramic image of a rope, the device comprising:

[0030] An acquisition module, used for acquiring multi-directional images of the rope to be tested;

[0031] a segmentation module, configured to segment the multi-directional image to obtain a rope image in the multi-directional image;

[0032] a correction module for correcting the coordinates of each pixel in each rope image according to a pre-fitted vertical coordinate transformation relationship and a pre-fitted horizontal coordinate transformation relationship to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; and determining the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image;

[0033] a judgment module, configured to judge whether t is less than a direction number threshold, and obtain a first judgment result, wherein the direction number threshold is greater than or equal to 3;

[0034] an iterative module, configured to, if the first judgment result indicating that t is less than a direction number threshold, determine, based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the t-th direction, the first weight in the t-th direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1-th direction, and the second weight in the t+1-th direction, a pixel value of each pixel in the t-th overlapping area after the first to-be-overlapped area in the corrected rope image in the t-th direction overlaps with the second to-be-overlapped area in the corrected rope image in the t+1-th direction; wherein t is a positive integer;

[0035] and a display module configured to display the fused rope panoramic image based on the pixel values ​​of the pixels in the overlapping areas of the corrected rope images and the pixel values ​​of the corrected coordinates of the pixels in the non-overlapping areas if the first judgment result indicates that t is not less than the direction number threshold.

[0036] In a third aspect, the present application provides a computing device, including a memory and a processor;

[0037] One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method as described in any one of the first aspects.

[0039] It can be seen from the above technical solution that this application has at least the following beneficial effects:

[0040] In the present application, the processing device obtains a multi-directional image of the rope to be measured, segments the multi-directional image to obtain rope images in the multi-directional image, corrects the coordinates of each pixel in each rope image according to a pre-fitted vertical coordinate transformation relationship and a horizontal coordinate transformation relationship, obtains the corrected coordinates of each pixel in the corresponding corrected rope image, determines the pixel values of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image, and displays the fused rope panoramic image according to the pixel values of each pixel in the overlapping area of each corrected rope image and the pixel values of the corrected coordinates of each pixel in the non-overlapping area. At present, during the detection process outside the rope, manual detection methods such as visual method are generally used. This detection method requires the detection personnel to observe around the rope for one round, which not only consumes manpower and is low in efficiency, but also is prone to false positives or false negatives caused by observation failure. It can be seen that the present application generates an image of the rope, obtains the pixel values of each pixel in the image, and fuses to obtain a panoramic image around the rope for one round, and then the detection personnel detects the rope. It can be seen that the scheme in the present application can facilitate the detection personnel to detect the rope and improve the detection efficiency.

[0041] It should be understood that the description of technical features, technical solutions, advantages or similar language in the present application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it can be understood that the description of a feature or advantage means that the specific technical feature, technical solution or advantage is included in at least one embodiment. Therefore, the description of technical features, technical solutions or advantages in the specification does not necessarily refer to the same embodiment. Further, the technical features, technical solutions and advantages described in the embodiments can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or advantages of a particular embodiment. In other embodiments, additional technical features and advantages can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0043] Figure 2 A schematic diagram of an adjustable camera and light source support device provided for an embodiment of the present application;

[0044] Figure 3 A flowchart of a rope panoramic image generation method provided for an embodiment of the present application;

[0045] Figure 4A schematic diagram of a device for generating a panoramic image of a rope provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The terms "first", "second" and "third" in this application specification and the accompanying drawings are used to distinguish different objects rather than to limit a specific order.

[0048] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0049] At present, manual inspection methods such as visual inspection are generally used in the process of inspecting the outside of the rope. This inspection method requires the inspector to walk around the rope for observation. It is not only manpower-consuming and inefficient, but also prone to false alarms or missed alarms due to inadequate observation.

[0050] In light of this, an embodiment of the present application provides a method for generating a panoramic image of a rope. In this method, a processing device acquires multi-directional images of the rope to be tested, segments the multi-directional images, and obtains rope images within the multi-directional images. The coordinates of each pixel in each rope image are corrected based on pre-fitted vertical and horizontal coordinate transformation relationships to obtain the corrected coordinates of each pixel in the corresponding corrected rope image. Based on the coordinates of each pixel in each rope image, the pixel values ​​of the corrected coordinates of each pixel in the corresponding corrected rope image are determined. The fused panoramic image of the rope is displayed based on the pixel values ​​of each pixel in the overlapping areas of the corrected rope images and the pixel values ​​of each pixel in the non-overlapping areas of the corrected rope images. Thus, the present application generates an image of the rope, obtains the pixel values ​​of each pixel in the image, and fuses the images to obtain a panoramic image encompassing the rope. The rope is then inspected by an inspector. This solution can facilitate rope inspection by inspectors and improve inspection efficiency.

[0051] In order to make the technical solution of this application clearer and easier to understand, the following describes an application scenario provided by the embodiment of this application by introducing the generation process of the above-mentioned rope panoramic image. Figure 1 As shown in the figure, this figure is a schematic diagram of an application scenario provided by an embodiment of the present application.

[0052] In this scenario, an aluminum profile support 11 is placed in the target area of ​​the rope to be detected, an adjustable camera and light source bracket 10 is installed on the aluminum profile support, and a network port industrial camera 9 and a coaxial light source 8 are installed on the adjustable camera and light source bracket 10, wherein the number of network port industrial cameras 9 is greater than or equal to 3. In the application scenario provided in the embodiment of the present application, the number of network port industrial cameras 9 is set to 4, and the bracket position is adjusted to ensure that the distance between each network port industrial camera 9 and the rope 7 is approximately equal. The network port industrial camera 9 is connected to the data acquisition card 6 via the RJ45 network port for collecting the original image of the rope. All the original images of the rope are collected by the data acquisition card 6 and transmitted to the industrial computer 2. The original image is processed by the core algorithm and the processing results are stored. The industrial computer 2 is connected to the light source controller 4 via the RS232 interface and performs corresponding control actions on the light source controller 4. The user end at the industrial site can control the system through the user control panel 1. The AC power supply 5 is connected to the industrial computer 2 and the light source controller 4 through the power cord to power each module.

[0053] To ensure proper monitoring of the rope's status during movement, and to avoid excessive computational overhead and data redundancy caused by the network-ported industrial camera 9 due to excessively high image acquisition frequencies, or missed areas due to infrequent image acquisition frequencies, the rope motion control system 3 uses feedback to control the image acquisition frequency of the network-ported industrial camera 9. This control system transmits the current rope travel distance to the industrial computer 2 in real time. When the rope reaches a preset distance, the industrial computer 2 controls the network-ported industrial camera 9 to begin capturing images of the rope.

[0054] The following is a description of the operation process of the entire system.

[0055] The user control panel 1 starts the system, the AC power supply 5 supplies power to the entire system, and the rope motion control system 3 transmits the rope motion signal to the industrial computer 2. When the rope moves a certain distance, the industrial computer 2 controls the network port industrial camera 2 to capture images. The data acquisition card 6 stores the images captured by the network port industrial camera 9 and transmits them to the industrial computer 2. The industrial computer 2 performs calculations on the captured images and stores the results in a designated storage area.

[0056] The adjustable camera and light source bracket 10 is described in detail below. Figure 2As shown, this figure is a schematic diagram of an adjustable camera and light source bracket device provided in an embodiment of the present application. The visual module is mainly composed of multiple individual camera modules installed on an aluminum profile bracket through a bracket support 17. The figure shows one of the cameras and light source bracket devices. The bracket support 17 has two mounting holes connected to the aluminum profile bracket through T-bolts. In addition, there are two guide rails connected to the bracket adapter block 16 through screws, which can enable the network port industrial camera 9 to achieve movement adjustment and rotation adjustment within a certain range. There are eight cylindrical countersunk holes on the bracket adapter block 16, two in a group, and each group is separated by a certain distance. The bracket adapter block 16 is connected to the light source adapter block 13 through the countersunk holes. The light source adapter block 13 is connected to the light source adapter frame 12 through a hexagon socket head screw. The light source adapter frame 12 is connected to the coaxial light source 8 through bolts to form a light source module. The four groups of countersunk holes can be used to adjust the position of the light source and the camera, and adjust the lighting effect of the light source. The bracket adapter block 16 has two threaded holes connected to the camera support 15 through bolts, the camera support 15 is connected to the camera adapter block 14 through screws, and the camera adapter block 14 is connected to the network port industrial camera 9 through hexagon socket head screws.

[0057] In order to make the technical solution of this application clearer and easier to understand, the following describes a method for generating a panoramic image of a rope provided by an embodiment of this application in conjunction with the accompanying drawings. Figure 3 As shown in FIG, this figure is a flow chart of a method for generating a panoramic image of a rope provided in an embodiment of the present application. The method for generating a panoramic image of a rope can be executed by a processing device, and the method includes:

[0058] S301: The processing device obtains multi-directional images of the rope to be tested.

[0059] The number of multi-directional images is greater than or equal to 3. In the embodiment of the present application, the number of multi-directional images is set to 4.

[0060] Among them, the multi-directional images include a first image in the first direction, a second image in the second direction, a third image in the third direction and a fourth image in the fourth direction. The first direction is opposite to the third direction, the second direction is opposite to the fourth direction, the first direction is perpendicular to the second direction, and the third direction is perpendicular to the fourth direction.

[0061] The processing device collects images of the rope through the network port industrial camera 9. The rope images in the first direction, second direction, third direction and fourth direction are collected in order to obtain all images of the rope around, provide support for subsequent operations, and facilitate the inspection personnel to conduct rope inspections in the later stage.

[0062] S302: The processing device segments the multi-directional image.

[0063] The rope background image is stored in the data acquisition card 6 in advance by the web port industrial camera 9 without equipping the rope, after equipping the rope, the web port industrial camera 9 starts to collect images, the image collected by each web port industrial camera 9 is subjected to difference operation with the background image stored in advance by each web port industrial camera 9 to separate the rope image in the foreground, and then the foreground image is subjected to binarization, opening operation, closing operation and the like. After the pre-processing is completed, the minimum circumscribed rectangle of the contour in the rope image is detected, the detected rectangular region is screened, the rope region is separated, and the rope image in the multi-direction image is obtained, and every interval preset distance, the multi-direction image of the rope to be detected is obtained. In this way, only the rope image can be focused on, and the surrounding environment will not be affected.

[0064] In S303, the processing device corrects the coordinates of each pixel in each rope image according to the pre-fitted coordinate transformation relationship in the vertical direction and the coordinate transformation relationship in the horizontal direction.

[0065] Since the model of the rope is similar to a cylinder, the edge of the separated rope image will have an image distortion problem, and the conventional global homography transformation matrix is also difficult to describe the transformation relationship between the images captured by adjacent industrial cameras, therefore, a checkerboard is used as a calibration object and completely adheres to the curved surface to be corrected, and a quadratic surface expression is calculated according to the coordinate distribution rule of the corner points of the checkerboard to fit the curved surface.

[0066]

[0067] Wherein x k and y k are the horizontal and vertical coordinates of the point to be fitted, x Tk and y Tk represent the horizontal and vertical coordinates of the kth corner point of the upper checkerboard, x Bk and y Bk represent the horizontal and vertical coordinates of the kth corner point of the lower checkerboard; a0 is the fourth coefficient fitted in advance, b0 is the fifth coefficient fitted in advance, and c0 is the sixth coefficient fitted in advance. A number of groups of points are brought into the equation to obtain an over-determined equation group (2). The optimal solution of the quadratic polynomial coefficients is obtained by solving the over-determined equation group by the least square method. The original image is subjected to inverse projection transformation correction according to the obtained quadratic polynomial.

[0068] The coordinate transformation relationship in the vertical direction is represented by formula (3):

[0069]

[0070] Wherein y″ i is the vertical coordinate of the i th pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, and Y0 is the extreme value corresponding to the extreme point X0; x iis the horizontal coordinate of the i-th pixel before correction, y i is the vertical coordinate of the i-th pixel before correction.

[0071] The distribution of pixel equivalents in the horizontal direction is similar to that in the vertical direction, but the pixel equivalents in the horizontal direction are continuously changing. Here, each checkerboard square is regarded as a unit, and the integral of the function D(x) of the pixel equivalent with respect to the horizontal coordinate of the pixel in this unit is divided by the length of this unit to obtain the pixel equivalent of this area. The specific expression is:

[0072]

[0073]

[0074] Substituting several points on the chessboard into the above equations, we obtain the system of equations shown in Equation (5). Solving the overdetermined system of equations using the least squares method yields the optimal coefficients for the fitted quadratic polynomial. Similarly to the vertical correction, after obtaining the optimal coefficients for the fitted quadratic polynomial, we perform back-projection correction on the original image in the x-direction.

[0075] The horizontal coordinate transformation relationship is represented by formula (6):

[0076]

[0077] Where x″ i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the first coefficient of the pre-fitting, b1 is the second coefficient of the pre-fitting, c1 is the third coefficient of the pre-fitting, and x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.

[0078] Therefore, the corrected coordinates (x″) of each pixel in the corresponding corrected rope image are obtained. i ,y″ i ).

[0079] Correcting the coordinates of each pixel in each rope image can improve measurement accuracy and reduce image coordinate errors caused by factors such as device angle, lens distortion, and shooting environment. The corrected coordinates are more accurate, making subsequent operations more reliable.

[0080] S304: The processing device determines the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image according to the coordinate of each pixel in each rope image.

[0081] Determining, based on the coordinates of each pixel in each rope image, the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image, including:

[0082] q(x″ i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl (8)

[0083] Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the upper right adjacent pixel of the i-th pixel before correction, q tl is the pixel value of the upper left adjacent pixel of the i-th pixel before correction, q bl is the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the vertical coordinate of the i-th pixel before correction.

[0084] In image analysis processes such as edge detection, feature extraction, and image recognition, accurate pixel coordinates and values ​​are essential for ensuring accuracy. Corrected images reduce errors caused by distortion, making the analysis process more accurate and reliable, and improving the precision and credibility of the results. In applications where multiple rope images need to be fused or spliced ​​together to form a more complete scene, corrected pixel coordinates and values ​​are essential for seamless fusion and splicing. By ensuring consistent pixel coordinates between images, stitching gaps are eliminated, improving the visual quality and practicality of the fused image.

[0085] S305: The processing device determines whether t is less than 4.

[0086] The processing device determines whether t is less than a direction number threshold, where the direction number threshold is greater than or equal to 3. In the embodiment of the present application, the direction number threshold is set to 4, and a first judgment result is obtained;

[0087] If the first judgment result indicates that t is less than 4, execute S306;

[0088] If the first judgment result indicates that t is not less than 4, execute S307.

[0089] S306: The processing device determines the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the t+1th direction.

[0090] The processing device determines, based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the t-th direction, the first weight in the t-th direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1-th direction, and the second weight in the t+1-th direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t-th direction.

[0091] The first area to be overlapped is an area used to overlap with the next direction, the second area to be overlapped is an area used to overlap with the previous direction, and t is a positive integer.

[0092] In some embodiments, t is set to 1, and the processing device determines the pixel value of each pixel in the first overlapping area after the first to-be-overlapped area in the corrected rope image in the first direction overlaps with the second to-be-overlapped area in the corrected rope image in the second direction based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the first direction, the first weight in the first direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the second direction, and the second weight in the second direction.

[0093] S307 : The processing device displays the fused rope panoramic image according to the pixel value of each pixel in the overlapping area of ​​each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area.

[0094] A conversion relationship between the coordinate points of the first area to be overlapped in the corrected rope image in the first direction and the coordinate points of the second area to be overlapped in the corrected rope image in the second direction is established. Since the rope has been unfolded into a plane in the preprocessing stage, the corresponding relationship between the first area to be overlapped in the corrected rope image in the first direction and the second area to be overlapped in the corrected rope image in the second direction is shown in formula (7):

[0095]

[0096] Wherein, (x, y) is the coordinate point of the first area to be overlapped in the corrected rope image in the first direction, (x ′ ,y′ is a coordinate point of the second to-be-overlapped region in the corrected rope image of the second direction, h 11 is a first parameter, h 12 is a second parameter, h 13 is a third parameter, h 21 is a fourth parameter, h 22 is a fifth parameter, h 23 is a sixth parameter, h 31 is a seventh parameter, h 32 is an eighth parameter, h 33 is a ninth parameter.

[0097] After obtaining the matched points of the image of the tth direction and the image of the (t+1)th direction through formula (9), the RANSAC algorithm is used to discard abnormal matched points as outliers.

[0098] Formula (10) is a calculation of parameters h 11 , h 12 , h 13 , h 21 , h 22 , h 23 , h 31 , h 32 , h 33 .

[0099]

[0100] At least 4 sets of coordinate points are required to solve all parameters to solve the homographic transformation matrix, but more than 4 points are used in the calculation process to solve the homographic transformation matrix by the least square method.

[0101] After the processing device maps the to-be-stitched images onto the same canvas, the pixel weighted average method is used for image fusion. According to the gray information of the two images, the pixel weighted average method respectively assigns a weight value to the gray values of the two images at the same pixel point, and the gray value of the fused image is the weighted sum of the gray values of the two images. If it is a color image, the processing device repeats the above operation on three channels to obtain the fused gray value on three channels. Thus, the rope panoramic image is obtained. The fusion formula is shown in formula (11):

[0102]

[0103] Wherein, f(x″,y″) is the fusion weight function of the coordinate points of the first area to be overlapped in the corrected rope image in the first direction and the coordinate points of the second area to be overlapped in the corrected rope image in the second direction, f1(x″,y″) is the set function of the coordinate points of the first area to be overlapped in the corrected rope image in the first direction, f2(x″,y″) is the set function of the coordinate points of the second area to be overlapped in the corrected rope image in the second direction, ω1 is the first fusion parameter, and ω2 is the second fusion parameter.

[0104] After fusing the coordinate points of the first to-be-overlapped area in the corrected rope image in the t-th direction with the coordinate points of the second to-be-overlapped area in the corrected rope image in the t+1-th direction, a rope panoramic image is obtained. The rope panoramic image is sent to a user terminal, and a marking result of the rope panoramic image by the user is received. If the marking result indicates that the health status score of the rope to be tested is lower than a scoring threshold, it indicates that there is a problem with the marking result at the position of the rope panoramic image and that the position needs to be repaired. The corresponding position of the rope panoramic image in the rope to be tested is recorded. If the marking result indicates that the health status score of the rope to be tested is higher than the scoring threshold, the marking result of the next rope panoramic image is detected.

[0105] Then, a color difference detection is performed on the rope panoramic image to obtain a detection result. If the detection result indicates that there is a target area in the rope panoramic image with a color difference greater than a preset color difference threshold, it means that there is a large color difference between this position and other positions in the rope panoramic image, and this position needs to be focused on. Then, the target area is highlighted to further facilitate user observation. If the detection result indicates that there is no target area in the rope panoramic image with a color difference greater than the preset color difference threshold, the next step of detection is performed.

[0106] Based on the above description, an embodiment of the present application provides a method for generating a panoramic image of a rope. In this method, a processing device obtains multi-directional images of the rope to be tested, segments the multi-directional images, and obtains rope images within the multi-directional images. The coordinates of each pixel in each rope image are corrected based on pre-fitted vertical and horizontal coordinate transformation relationships to obtain the corrected coordinates of each pixel in the corresponding corrected rope image. Based on the coordinates of each pixel in each rope image, a pixel value for each corrected coordinate in the corresponding corrected rope image is determined. The fused panoramic image of the rope is displayed based on the pixel values ​​of each pixel in the overlapping areas of the corrected rope images and the pixel values ​​of each pixel in the non-overlapping areas of the corrected rope images. Thus, the present application generates an image of the rope, obtains the pixel values ​​of each pixel in the image, and fuses the images to obtain a panoramic image encompassing the rope. The rope is then inspected by an inspector. This solution can facilitate rope inspection by inspectors and improve inspection efficiency.

[0107] The embodiment of the present application also provides a schematic diagram of a device for generating a panoramic image of a rope, such as Figure 4 As shown in the figure, this figure is a schematic diagram of a device for generating a panoramic image of a rope provided in an embodiment of the present application, and the device for generating a panoramic image of a rope includes: Figure 4 The illustrated example includes an acquisition module 401 , a segmentation module 402 , a correction module 403 , a judgment module 404 , an iteration module 405 and a display module 406 .

[0108] The present application provides a device for generating a panoramic image of a rope, the device comprising:

[0109] An acquisition module 401 is used to acquire multi-directional images of the rope to be tested;

[0110] a segmentation module 402 for segmenting the multi-directional image to obtain a rope image in the multi-directional image;

[0111] The correction module 403 is configured to correct the coordinates of each pixel in each rope image based on the pre-fitted vertical coordinate transformation relationship and the pre-fitted horizontal coordinate transformation relationship to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; and determine the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image based on the coordinates of each pixel in the rope image.

[0112] A judgment module 404 is configured to judge whether t is less than a direction number threshold, and obtain a first judgment result, wherein the direction number threshold is greater than or equal to 3;

[0113] an iterative module 405 configured to determine, if the first judgment result indicating that t is less than a direction number threshold, a pixel value of each pixel in a tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the tth direction, based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction; wherein t is a positive integer;

[0114] The display module 406 is configured to display the fused rope panoramic image based on the pixel values ​​of each pixel in the overlapping area of ​​each corrected rope image and the pixel values ​​of the corrected coordinates of each pixel in the non-overlapping area if the first judgment result indicates that t is not less than the direction number threshold.

[0115] Optionally, the coordinate transformation relationship in the horizontal direction is represented by the following formula:

[0116]

[0117] Among them, x" i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the first coefficient of the pre-fitting, b1 is the second coefficient of the pre-fitting, c1 is the third coefficient of the pre-fitting, and x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.

[0118] Optionally, the coordinate transformation relationship in the vertical direction is represented by the following formula:

[0119]

[0120] Among them, y i is the ordinate of the i-th pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, Y0 is the extreme value corresponding to the extreme point X0; a0 is the pre-fitted fourth coefficient, b0 is the pre-fitted fifth coefficient, c0 is the pre-fitted sixth coefficient; x i is the horizontal coordinate of the i-th pixel before correction, y i is the vertical coordinate of the i-th pixel before correction.

[0121] Optionally, determining the pixel value of the corrected coordinate of each pixel in each corresponding corrected rope image according to the coordinate of each pixel in each rope image includes:

[0122] q(x″ i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl

[0123] Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the upper right adjacent pixel of the i-th pixel before correction, q tl is the pixel value of the upper left adjacent pixel of the i-th pixel before correction, q blis the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the vertical coordinate of the i-th pixel before correction.

[0124] Optionally, the method further includes:

[0125] receiving a marking result of the rope panoramic image from a user;

[0126] If the marking result indicates that the health status score of the rope to be tested is lower than a score threshold, the corresponding position of the rope panoramic image in the rope to be tested is recorded.

[0127] Optionally, the acquiring of multi-directional images of the rope to be tested includes:

[0128] At every preset distance, multi-directional images of the rope to be tested are acquired.

[0129] The virtual object allocation device according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above-mentioned other operations and / or functions of each module / unit of the virtual object allocation device are to achieve Figure 3 For the sake of brevity, the corresponding processes of the various methods in the illustrated embodiments are not described again here.

[0130] The present application also provides a computing device. Figure 5 As shown, this figure is a schematic diagram of a computing device provided by an embodiment of the present application, and the computing device 700 includes a bus 701, a processor 702, a communication interface 703 and a memory 704. The processor 702, the memory 704 and the communication interface 703 communicate with each other via the bus 701.

[0131] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0132] The processor 702 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0133] The communication interface 703 is used for communicating with the outside.

[0134] The memory 704 may include volatile memory, such as random access memory (RAM), or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0135] The memory 704 stores executable codes, and the processor 702 executes the executable codes to perform the aforementioned method for generating a panoramic image of a rope.

[0136] Specifically, in the implementation Figure 4 In the case of the embodiment shown, and Figure 4 When each module or unit of the virtual object allocation device described in the embodiment is implemented by software, Figure 4 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 704. The processor 702 executes the program code corresponding to each unit stored in the memory 704 to perform the aforementioned method for generating a panoramic image of a rope.

[0137] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, or magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the aforementioned method for generating a panoramic rope image.

[0138] The present application also provides a computer program product comprising one or more computer instructions that, when loaded and executed on a computing device, fully or partially generate the process or function described in the present application.

[0139] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0140] When the computer program product is executed by a computer, the computer performs any of the aforementioned methods for generating a panoramic image of a rope. The computer program product may be a software installation package that can be downloaded and executed on a computer when any of the aforementioned methods for generating a panoramic image of a rope is desired.

[0141] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0142] The above description is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a panoramic image of a rope, characterized in that: The method comprises: Acquire multi-directional images of the rope to be tested; Segmenting the multi-directional image to obtain a rope image in the multi-directional image; Correcting the coordinates of each pixel in each rope image according to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; determining pixel values ​​of the corrected coordinates of the pixels in the corresponding corrected rope images according to the coordinates of the pixels in the rope images; Determine whether t is less than a direction number threshold, obtaining a first determination result, wherein the direction number threshold is greater than or equal to 3; wherein t is the number of directions of the rope image; If the first judgment result indicates that t is less than the direction number threshold, then determining the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the tth direction based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction; wherein t is a positive integer; If the first judgment result indicates that t is not less than the direction number threshold, the fused rope panoramic image is displayed according to the pixel value of each pixel in the overlapping area of ​​each corrected rope image and the pixel value of the corrected coordinate of each pixel in the non-overlapping area.

2. The method according to claim 1, characterized in that The horizontal coordinate transformation relationship is represented by the following formula: in, is the horizontal coordinate of the i-th pixel after correction, is the horizontal coordinate of the first pixel before correction, is the offset distance of the nth pixel in the horizontal direction, is the first coefficient of the pre-fitting, is the second coefficient of the pre-fitting, is the third coefficient of the pre-fitting, is the horizontal coordinate of the nth pixel before correction, is the extreme point of the quadratic polynomial.

3. The method according to claim 1, characterized in that The coordinate transformation relationship in the vertical direction is represented by the following formula: in, is the ordinate of the i-th pixel after correction, is the height of the image, is the extreme point of the quadratic polynomial, is the extreme point The corresponding extreme value; is the fourth coefficient of the pre-fitting, is the fifth coefficient of the pre-fitting, is the sixth coefficient of the pre-fitting; is the horizontal coordinate of the i-th pixel before correction, is the vertical coordinate of the i-th pixel before correction.

4. The method according to claim 1, wherein Determining the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image according to the coordinate of each pixel in each rope image includes: in, is the pixel value of the i-th pixel after correction, is the pixel value of the pixel adjacent to the upper right of the i-th pixel before correction, is the pixel value of the upper left adjacent pixel of the i-th pixel before correction, is the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, is the horizontal coordinate of the i-th pixel before correction, is the vertical coordinate of the i-th pixel before correction.

5. The method according to claim 1, wherein The method further comprises: receiving a marking result of the rope panoramic image from a user; If the marking result indicates that the health status score of the rope to be tested is lower than a score threshold, the corresponding position of the rope panoramic image in the rope to be tested is recorded.

6. The method according to claim 1, characterized in that The method further comprises: Performing color difference detection on the panoramic image of the rope to obtain a detection result; If the detection result indicates that there is a target area in the rope panoramic image whose color difference is greater than a preset color difference threshold, the target area is highlighted.

7. The method according to claim 1, characterized in that The obtaining of multi-directional images of the rope to be tested comprises: At every preset distance, multi-directional images of the rope to be tested are acquired.

8. A device for generating a panoramic image of a rope, characterized in that: The device comprises: An acquisition module, used for acquiring multi-directional images of the rope to be tested; a segmentation module, configured to segment the multi-directional image to obtain a rope image in the multi-directional image; a correction module for correcting the coordinates of each pixel in each rope image according to a pre-fitted vertical coordinate transformation relationship and a pre-fitted horizontal coordinate transformation relationship to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; and determining the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image; a judgment module, configured to judge whether t is less than a direction number threshold, and obtain a first judgment result, wherein the direction number threshold is greater than or equal to 3; wherein t is the number of directions of the rope image; an iterative module, configured to, if the first judgment result indicating that t is less than a direction number threshold, determine, based on the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the t-th direction, the first weight in the t-th direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1-th direction, and the second weight in the t+1-th direction, a pixel value of each pixel in the t-th overlapping area after the first to-be-overlapped area in the corrected rope image in the t-th direction overlaps with the second to-be-overlapped area in the corrected rope image in the t+1-th direction; wherein t is a positive integer; and a display module configured to display the fused rope panoramic image based on the pixel values ​​of the pixels in the overlapping areas of the corrected rope images and the pixel values ​​of the corrected coordinates of the pixels in the non-overlapping areas if the first judgment result indicates that t is not less than the direction number threshold.

9. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

Images