Rope panoramic image generation method and device, equipment and storage medium
By generating panoramic images of the rope, the problem of low efficiency in existing rope external detection is solved, and efficient and accurate rope detection is achieved.
Patent Information
- Application Number
- CN202411267073.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-11
AI Technical Summary
Existing external rope detection methods are inefficient, prone to false alarms or missed alarms, and require significant manpower.
By acquiring multi-directional images of the rope, segmentation and coordinate correction are performed. Using pre-fitted coordinate transformation relationships in the vertical and horizontal directions, a panoramic image of the rope is generated, and pixel value fusion is performed to display the panoramic image of the rope.
It improved the efficiency of rope detection, reduced false alarms and missed alarms, and saved human resources.
Smart Images

Figure CN119048367A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating a panoramic image of a rope. Background Art
[0002] The health status of a rope is used to characterize the safety and reliability of the rope. In the fields of industry, construction, transportation, etc., ropes are important load-bearing and connection equipment, so it is of great significance to detect the health status of ropes. Generally, the health status of ropes can be analyzed by testing the inside and outside of the rope.
[0003] At present, manual inspection methods such as visual inspection are generally used in the process of inspecting the outside of the rope. This inspection method requires the inspector to observe the rope around once, which is not only labor-intensive and inefficient, but also prone to false alarms or missed alarms due to inadequate observation.
[0004] Therefore, the detection efficiency of the above-mentioned detection scheme for the health status of the rope is low. Summary of the invention
[0005] The present application provides a method, device, equipment and storage medium for generating a panoramic image of a rope, which can facilitate inspection personnel to inspect the rope and improve inspection efficiency.
[0006] In order to achieve the above purpose, this application adopts the following technical solutions:
[0007] In a first aspect, the present application provides a method for generating a panoramic image of a rope, comprising:
[0008] Acquire multi-directional images of the rope to be tested;
[0009] Segmenting the multi-directional image to obtain a rope image in the multi-directional image;
[0010] According to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship, the coordinates of each pixel in each rope image are corrected to obtain the corrected coordinates of each pixel in the corresponding corrected rope image;
[0011] According to the coordinates of each pixel in each rope image, determine the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image;
[0012] Determine whether t is less than a direction number threshold, and obtain a first determination result, wherein the direction number threshold is greater than or equal to 3;
[0013] If the first judgment result indicates that t is less than the direction number threshold, then according to the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the t-th direction, the first weight in the t-th direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1-th direction, and the second weight in the t+1-th direction, the pixel value of each pixel in the t-th overlapping area after the first to-be-overlapped area in the corrected rope image in the t-th direction overlaps with the second to-be-overlapped area in the corrected rope image in the t+1-th direction is determined; wherein t is a positive integer;
[0014] If the first judgment result indicates that t is not less than the direction number threshold, the fused rope panoramic image is displayed according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area.
[0015] In some possible implementations, the coordinate transformation relationship in the horizontal direction is represented by the following formula:
[0016]
[0017] Among them, x″ i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the pre-fitted first coefficient, b1 is the pre-fitted second coefficient, c1 is the pre-fitted third coefficient, x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.
[0018] In some possible implementations, the coordinate transformation relationship in the vertical direction is represented by the following formula:
[0019]
[0020] Among them, y″ i is the ordinate of the ith pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, Y0 is the extreme value corresponding to the extreme point X0; a0 is the fourth coefficient of the pre-fitting, b0 is the fifth coefficient of the pre-fitting, and c0 is the sixth coefficient of the pre-fitting; x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
[0021] In some possible implementations, determining the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image according to the coordinate of each pixel in each rope image includes:
[0022] q(x″i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl
[0023] Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the pixel adjacent to the upper right of the i-th pixel before correction, q tl is the pixel value of the pixel adjacent to the upper left of the i-th pixel before correction, q bl is the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
[0024] In some possible implementations, the method further includes:
[0025] receiving a marking result of the panoramic image of the rope from a user;
[0026] If the marking result indicates that the health status score of the rope to be tested is lower than a score threshold, the position corresponding to the rope panoramic image in the rope to be tested is recorded.
[0027] In some possible implementations, the acquiring of multi-directional images of the rope to be tested includes:
[0028] At every preset distance, multi-directional images of the rope to be tested are obtained.
[0029] In a second aspect, the present application provides a device for generating a panoramic image of a rope, the device comprising:
[0030] An acquisition module, used for acquiring multi-directional images of the rope to be tested;
[0031] A segmentation module, used for segmenting the multi-directional image to obtain a rope image in the multi-directional image;
[0032] A correction module, for correcting the coordinates of each pixel in each rope image according to the pre-fitted coordinate transformation relationship in the vertical direction and the coordinate transformation relationship in the horizontal direction, to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; and determining the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image;
[0033] A judgment module, used to judge whether t is less than a direction number threshold, and obtain a first judgment result, wherein the direction number threshold is greater than or equal to 3;
[0034] an iterative module, configured to determine, if the first judgment result representation t is less than a direction number threshold, the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the tth direction, according to the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction; wherein t is a positive integer;
[0035] A display module is used to display the fused rope panoramic image according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinate of each pixel in the non-overlapping area if the first judgment result representation t is not less than the direction number threshold.
[0036] In a third aspect, the present application provides a computing device, including a memory and a processor;
[0037] One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.
[0038] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method as described in any one of the first aspects.
[0039] It can be seen from the above technical solution that the present application has at least the following beneficial effects:
[0040] In the present application, the processing device obtains a multi-directional image of the rope to be tested, segments the multi-directional image, obtains the rope image in the multi-directional image, corrects the coordinates of each pixel in each rope image according to the pre-fitted vertical coordinate transformation relationship and the horizontal coordinate transformation relationship, obtains the corrected coordinates of each pixel in the corresponding corrected rope image, and then determines the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image, and displays the fused rope panoramic image according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area. At present, in the process of detecting the outside of the rope, manual detection methods such as visual method are generally used. This detection method requires the detection personnel to observe around the rope for a week, which is not only labor-intensive and inefficient, but also prone to false alarms or missed alarms caused by inadequate observation. It can be seen that the present application generates an image for the rope, obtains the pixel value of each pixel in the image, and fuses it to obtain a panoramic image around the rope, and then the detection personnel detect the rope. It can be seen that the solution in the present application can facilitate inspection personnel to inspect ropes and improve inspection efficiency.
[0041] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;
[0043] Figure 2 A schematic diagram of an adjustable camera and light source support device provided in an embodiment of the present application;
[0044] Figure 3 A flowchart of a method for generating a panoramic image of a rope provided in an embodiment of the present application;
[0045] Figure 4A schematic diagram of a device for generating a panoramic image of a rope provided in an embodiment of the present application;
[0046] Figure 5 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0047] The terms "first", "second", "third", etc. in the specification of this application and the accompanying drawings are used to distinguish different objects rather than to limit a specific order.
[0048] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0049] At present, manual inspection methods such as visual inspection are generally used in the process of inspecting the outside of the rope. This inspection method requires the inspector to observe the rope around once, which is not only labor-intensive and inefficient, but also prone to false alarms or missed alarms due to inadequate observation.
[0050] In view of this, an embodiment of the present application provides a method for generating a panoramic image of a rope, in which a processing device obtains a multi-directional image of a rope to be tested, segments the multi-directional image, obtains a rope image in the multi-directional image, corrects the coordinates of each pixel in each rope image according to a pre-fitted vertical coordinate transformation relationship and a horizontal coordinate transformation relationship, obtains the corrected coordinates of each pixel in each corresponding corrected rope image, and then determines the pixel value of the corrected coordinates of each pixel in each corresponding corrected rope image according to the coordinates of each pixel in each rope image, and displays the fused panoramic image of the rope according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area. It can be seen that the present application generates an image for the rope, obtains the pixel value of each pixel in the image, and fuses it to obtain a panoramic image around the rope, and then the rope is tested by the tester. It can be seen that the scheme in the present application can facilitate the tester to test the rope and improve the test efficiency.
[0051] In order to make the technical solution of the present application clearer and easier to understand, the following describes an application scenario provided by the embodiment of the present application by introducing the generation process of the above-mentioned rope panoramic image. Figure 1 As shown, the figure is a schematic diagram of an application scenario provided by an embodiment of the present application.
[0052] In this scenario, an aluminum profile support 11 is placed in the target area of the rope to be detected, an adjustable camera and light source bracket 10 is installed on the aluminum profile support, and a network port industrial camera 9 and a coaxial light source 8 are installed on the adjustable camera and light source bracket 10, wherein the number of network port industrial cameras 9 is greater than or equal to 3. In the application scenario provided in the embodiment of the present application, the number of network port industrial cameras 9 is set to 4, and the bracket position is adjusted to ensure that the distance between each network port industrial camera 9 and the rope 7 is approximately equal. The network port industrial camera 9 is connected to the data acquisition card 6 through the RJ45 network port for collecting the original image of the rope. All the original images of the rope are collected by the data acquisition card 6 and transmitted to the industrial computer 2. The original image is processed by the core algorithm and the processing result is stored. The industrial computer 2 is connected to the light source controller 4 through the RS232 interface, and the light source controller 4 is controlled accordingly. The user end at the industrial site can control the system through the user control panel 1, and the AC power supply 5 is connected to the industrial computer 2 and the light source controller 4 through the power cord, thereby powering each module.
[0053] In order to ensure that the rope state is properly monitored during the rope movement process, and to avoid excessive calculation and data redundancy caused by the network port industrial camera 9 due to too high a sampling frequency, or the network port industrial camera 9 causing missed detection areas due to too low a sampling frequency, the rope movement control system 3 is used to feedback control the sampling frequency of the network port industrial camera 9. The rope movement control system 3 transmits the current rope movement distance to the industrial computer 2 in real time. When the rope movement distance reaches the preset distance, the industrial computer 2 controls the network port industrial camera 9 to perform a rope sampling operation.
[0054] The following is a description of the operation process of the entire system.
[0055] The user control panel 1 starts the system, the AC power supply 5 supplies power to the entire system, the rope motion control system 3 transmits the rope motion signal to the industrial computer 2, when the rope moves a certain distance, the industrial computer 2 controls the network port industrial camera 2 to collect images, the data acquisition card 6 stores the images collected by the network port industrial camera 9 and transmits them to the industrial computer 2, the industrial computer 2 calculates and processes the collected images and stores the results in the designated storage area.
[0056] The adjustable camera and light source bracket 10 is described in detail below. Figure 2As shown, the figure is a schematic diagram of an adjustable camera and light source bracket device provided by an embodiment of the present application. The visual module is mainly composed of multiple individual camera modules installed on an aluminum profile bracket through a bracket support 17. The figure shows one of the cameras and light source bracket devices. The bracket support 17 has two mounting holes connected to the aluminum profile bracket through T-bolts, and two guide rails are connected to the bracket adapter block 16 through screws, so that the network port industrial camera 9 can achieve movement adjustment and rotation adjustment within a certain range. There are eight cylindrical countersunk holes on the bracket adapter block 16, two in a group, and each group is separated by a certain distance. The bracket adapter block 16 is connected to the light source adapter block 13 through the countersunk holes. The light source adapter block 13 is connected to the light source adapter frame 12 through the hexagon socket cylindrical head screw. The light source adapter frame 12 is connected to the coaxial light source 8 through bolts to form a light source module. The position of the light source and the camera can be adjusted through four groups of countersunk holes, and the lighting effect of the light source can be adjusted. The bracket adapter block 16 has two threaded holes connected to the camera support 15 through bolts, the camera support 15 is connected to the camera adapter block 14 through screws, and the camera adapter block 14 is connected to the network port industrial camera 9 through hexagon socket head screws.
[0057] In order to make the technical solution of the present application clearer and easier to understand, a method for generating a panoramic image of a rope provided by an embodiment of the present application is introduced below in conjunction with the accompanying drawings. Figure 3 As shown in FIG. 1 , the figure is a flow chart of a method for generating a panoramic image of a rope provided in an embodiment of the present application. The method for generating a panoramic image of a rope can be executed by a processing device, and the method includes:
[0058] S301: A processing device obtains multi-directional images of the rope to be tested.
[0059] The number of multi-directional images is greater than or equal to 3. In the embodiment of the present application, the number of multi-directional images is set to 4.
[0060] Among them, the multi-directional images include a first image in a first direction, a second image in a second direction, a third image in a third direction and a fourth image in a fourth direction. The first direction is opposite to the third direction, the second direction is opposite to the fourth direction, the first direction is perpendicular to the second direction, and the third direction is perpendicular to the fourth direction.
[0061] The processing device collects images of the rope through the network port industrial camera 9. The rope images in the first direction, the second direction, the third direction and the fourth direction are collected in order to obtain all images of the rope around, provide support for subsequent operations, and facilitate the subsequent rope inspection by the inspection personnel.
[0062] S302: The processing device segments the multi-directional image.
[0063] Before the rope is equipped, the network port industrial camera 9 is used to take a rope background image and store it in the data acquisition card 6. After the rope is equipped, the network port industrial camera 9 starts to collect images. The images collected by each network port industrial camera 9 are subjected to difference calculation with the background images pre-stored by each network port industrial camera 9 to separate the foreground rope image, and then the separated foreground image is subjected to binarization, opening operation, closing operation and other operations. After the preprocessing is completed, the minimum circumscribed rectangle of the contour in the rope image is detected, and the detected rectangular area is screened to separate the rope area to obtain the rope image in the multi-directional image, and the multi-directional image of the rope to be tested is obtained at every preset distance. In this way, only the rope image can be focused on without being affected by the surrounding environment.
[0064] S303: The processing device corrects the coordinates of each pixel in each rope image according to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship.
[0065] Since the model of the rope is approximately a cylinder, there will be image distortion problems at the edges of the segmented rope image. The conventional global homography transformation matrix is also difficult to describe the transformation relationship between images taken by adjacent industrial cameras. Therefore, a chessboard is used as a calibration object to fit the surface to be corrected completely. The quadratic surface expression is calculated through the coordinate distribution law of the chessboard corner points to fit the surface.
[0066]
[0067] where x k and k are the horizontal and vertical coordinates of the point to be fitted, respectively, Tk and Tk Respectively represent the horizontal and vertical coordinates of the kth corner point of the upper chessboard, x Bk and Bk Respectively represent the horizontal and vertical coordinates of the kth corner point of the lower chessboard; a0 is the fourth coefficient pre-fitted, b0 is the fifth coefficient pre-fitted, and c0 is the sixth coefficient pre-fitted. Substituting several groups of points into the equation can obtain an overdetermined system of equations (2). The optimal solution of the quadratic polynomial coefficients is obtained by solving the overdetermined system of equations using the least squares method. The original image is corrected by back-projection transformation based on the obtained quadratic polynomial.
[0068] The coordinate transformation relationship in the vertical direction is represented by formula (3):
[0069]
[0070] Among them, y″ i is the ordinate of the ith pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, and Y0 is the extreme value corresponding to the extreme point X0; iis the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
[0071] The distribution law of pixel equivalent in the horizontal direction is similar to that in the vertical direction, but the pixel equivalent in the horizontal direction is continuously changing. Here, each chessboard is regarded as a unit, and the integral of the function D(x) of the pixel equivalent with respect to the horizontal coordinate of the pixel in this unit is divided by the length of this unit to obtain the pixel equivalent of this area. The specific expression is:
[0072]
[0073]
[0074] Substitute several points in the chessboard into the above equations and get the equation group shown in equation (5). Solve the overdetermined equation group by the least square method to get the optimal coefficients of the fitting quadratic polynomial. After the correction in the vertical direction is obtained, the original image is corrected by back-projection transformation in the x direction.
[0075] The coordinate transformation relationship in the horizontal direction is represented by formula (6):
[0076]
[0077] Among them, x″ i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the pre-fitted first coefficient, b1 is the pre-fitted second coefficient, c1 is the pre-fitted third coefficient, x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.
[0078] Therefore, the corrected coordinates (x″) of each pixel in the corresponding corrected rope image are obtained. i ,y″ i ).
[0079] Correcting the coordinates of each pixel in each rope image can improve measurement accuracy and reduce image coordinate errors caused by factors such as equipment angle, lens distortion, and shooting environment. The corrected coordinates are more accurate, making subsequent operations more reliable.
[0080] S304: The processing device determines the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image according to the coordinate of each pixel in each rope image.
[0081] Determining pixel values of corrected coordinates of pixels in the corrected rope images according to the coordinates of pixels in the rope images, including:
[0082] q(x″ i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl (8)
[0083] Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the pixel adjacent to the upper right of the i-th pixel before correction, q tl is the pixel value of the pixel adjacent to the upper left of the i-th pixel before correction, q bl is the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
[0084] In the image analysis process, such as edge detection, feature extraction, image recognition, etc., accurate pixel coordinates and pixel values can ensure accuracy. The corrected image reduces the error caused by distortion, making the analysis process more accurate and reliable, and improving the accuracy and credibility of the analysis results. In applications where multiple rope images need to be fused or spliced to form a more complete scene, the corrected pixel coordinates and pixel values are the basis for achieving seamless fusion and splicing. By ensuring the consistency of pixel coordinates between images, the splicing gaps can be eliminated and the visual effect and practicality of the fused image can be improved.
[0085] S305: The processing device determines whether t is less than 4.
[0086] The processing device determines whether t is less than a direction number threshold, wherein the direction number threshold is greater than or equal to 3. In the embodiment of the present application, the direction number threshold is set to 4, and a first judgment result is obtained;
[0087] If the first judgment result indicates that t is less than 4, execute S306;
[0088] If the first judgment result indicates that t is not less than 4, S307 is executed.
[0089] S306: The processing device determines the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the t+1th direction.
[0090] The processing device determines the pixel value of each pixel in the tth overlapping area after the first area to be overlapped in the corrected rope image in the tth direction overlaps with the second area to be overlapped in the corrected rope image in the t+1th direction based on the pixel value of each pixel in the first area to be overlapped in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second area to be overlapped in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction.
[0091] The first area to be overlapped is an area used to overlap with the next direction, the second area to be overlapped is an area used to overlap with the previous direction, and t is a positive integer.
[0092] In some embodiments, t=1 is set, and the processing device determines the pixel value of each pixel in the first overlapping area after the first area to be overlapped in the corrected rope image in the first direction overlaps with the second area to be overlapped in the corrected rope image in the second direction according to the pixel value of each pixel in the first area to be overlapped in the corrected rope image in the first direction, the first weight in the first direction, the pixel value of each pixel in the second area to be overlapped in the corrected rope image in the second direction, and the second weight in the second direction.
[0093] S307: The processing device displays the fused rope panoramic image according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area.
[0094] A conversion relationship between the coordinate points of the first area to be overlapped in the corrected rope image in the first direction and the coordinate points of the second area to be overlapped in the corrected rope image in the second direction is established. Since the rope has been unfolded into a plane in the preprocessing stage, the corresponding relationship between the first area to be overlapped in the corrected rope image in the first direction and the second area to be overlapped in the corrected rope image in the second direction is shown in formula (7):
[0095]
[0096] Where (x, y) is the coordinate point of the first area to be overlapped in the corrected rope image in the first direction, (x ′ ,y′ ) is the coordinate point of the second area to be overlapped in the corrected rope image in the second direction, h 11 is the first parameter, h 12 is the second parameter, h 13 is the third parameter, h 21 is the fourth parameter, h 22 is the fifth parameter, h 23 is the sixth parameter, h 31 is the seventh parameter, h 32 is the eighth parameter, h 33 It is the ninth parameter.
[0097] After obtaining the matching points of the image in the t-th direction and the image in the t+1-th direction through formula (9), the abnormal matching point pairs are discarded as outliers through the RANSAC algorithm.
[0098] Formula (10) is the parameter h 11 、h 12 、h 13 、h 21 、h 22 、h 23 、h 31 、h 32 、h 33 Calculation.
[0099]
[0100] At least 4 sets of coordinate points are required to solve all parameters to solve the homography transformation matrix, but in the calculation process, far more than 4 points are used to solve the homography transformation matrix through the least squares method.
[0101] After the processing device maps the images to be stitched onto the same canvas, it uses the pixel weighted average method to fuse the images. The pixel weighted average method assigns weights to the grayscale values of the two images at the same pixel point according to the grayscale information of the two images themselves. The grayscale of the fused image is the weighted sum of the grayscale values of the two images. If it is a color image, the processing device repeats the above operation on three channels to obtain the fused grayscale on the three channels. Thus, the panoramic image of the rope is obtained. The fusion formula is shown in formula (11):
[0102]
[0103] Among them, f(x″,y″) is the fusion weight function of the coordinate points of the first area to be overlapped in the corrected rope image in the first direction and the coordinate points of the second area to be overlapped in the corrected rope image in the second direction, f1(x″,y″) is the coordinate point set function of the first area to be overlapped in the corrected rope image in the first direction, f2(x″,y″) is the coordinate point set function of the second area to be overlapped in the corrected rope image in the second direction, ω1 is the first fusion parameter, and ω2 is the second fusion parameter.
[0104] After the coordinate points of the first area to be overlapped in the corrected rope image in the t-th direction and the coordinate points of the second area to be overlapped in the corrected rope image in the t+1-th direction are merged, a rope panoramic image is obtained, the rope panoramic image is sent to the user end, and the user's marking result of the rope panoramic image is received; if the marking result indicates that the health status score of the rope to be tested is lower than the scoring threshold, it means that there is a problem with the marking result at the position of the rope panoramic image, and the position needs to be repaired, then the position corresponding to the rope panoramic image in the rope to be tested is recorded; if the marking result indicates that the health status score of the rope to be tested is higher than the scoring threshold, the marking result of the next rope panoramic image is detected.
[0105] Then, the rope panoramic image is subjected to color difference detection to obtain the detection result; if the detection result indicates that there is a target area in the rope panoramic image whose color difference is greater than the preset color difference threshold, it means that there is a large color difference between this position and other positions in the rope panoramic image, and it is necessary to focus on this position, then the target area is highlighted to further facilitate user observation; if the detection result indicates that there is no target area in the rope panoramic image whose color difference is greater than the preset color difference threshold, then the next step of detection is performed.
[0106] Based on the above description, the embodiment of the present application provides a method for generating a panoramic image of a rope, in which a processing device obtains a multi-directional image of a rope to be tested, segments the multi-directional image, obtains a rope image in the multi-directional image, corrects the coordinates of each pixel in each rope image according to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship, obtains the corrected coordinates of each pixel in the corresponding corrected rope image, and then determines the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image, and displays the fused panoramic image of the rope according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area. It can be seen that the present application generates an image for the rope, obtains the pixel value of each pixel in the image, and fuses it to obtain a panoramic image around the rope, and then the inspection personnel inspect the rope. It can be seen that the scheme in the present application can facilitate the inspection personnel to inspect the rope and improve the inspection efficiency.
[0107] The embodiment of the present application also provides a schematic diagram of a device for generating a panoramic image of a rope, such as Figure 4 As shown in the figure, this figure is a schematic diagram of a device for generating a panoramic image of a rope provided in an embodiment of the present application, and the device for generating a panoramic image of a rope includes: Figure 4 The illustrated components include an acquisition module 401 , a segmentation module 402 , a correction module 403 , a judgment module 404 , an iteration module 405 and a display module 406 .
[0108] The present application provides a device for generating a panoramic image of a rope, the device comprising:
[0109] An acquisition module 401 is used to acquire multi-directional images of the rope to be tested;
[0110] A segmentation module 402 is used to segment the multi-directional image to obtain a rope image in the multi-directional image;
[0111] The correction module 403 is used to correct the coordinates of each pixel in each rope image according to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; according to the coordinates of each pixel in each rope image, determine the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image;
[0112] A judgment module 404 is used to judge whether t is less than a direction number threshold, and obtain a first judgment result, wherein the direction number threshold is greater than or equal to 3;
[0113] an iterative module 405, for determining, if the first judgment result representation t is less than a direction number threshold, the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the tth direction, according to the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction; wherein t is a positive integer;
[0114] The display module 406 is used to display the fused rope panoramic image according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinate of each pixel in the non-overlapping area if the first judgment result representation t is not less than the direction number threshold.
[0115] Optionally, the coordinate transformation relationship in the horizontal direction is represented by the following formula:
[0116]
[0117] Among them, x" i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the pre-fitted first coefficient, b1 is the pre-fitted second coefficient, c1 is the pre-fitted third coefficient, x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.
[0118] Optionally, the coordinate transformation relationship in the vertical direction is represented by the following formula:
[0119]
[0120] Among them, y″ i is the ordinate of the ith pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, Y0 is the extreme value corresponding to the extreme point X0; a0 is the fourth coefficient of the pre-fitting, b0 is the fifth coefficient of the pre-fitting, and c0 is the sixth coefficient of the pre-fitting; x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
[0121] Optionally, determining the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image according to the coordinate of each pixel in each rope image includes:
[0122] q(x″ i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl
[0123] Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the pixel adjacent to the upper right of the i-th pixel before correction, q tl is the pixel value of the pixel adjacent to the upper left of the i-th pixel before correction, q blis the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
[0124] Optionally, the method further includes:
[0125] receiving a marking result of the panoramic image of the rope from a user;
[0126] If the marking result indicates that the health status score of the rope to be tested is lower than a score threshold, the position corresponding to the rope panoramic image in the rope to be tested is recorded.
[0127] Optionally, the acquiring of multi-directional images of the rope to be tested includes:
[0128] At every preset distance, multi-directional images of the rope to be tested are obtained.
[0129] The virtual object allocation device according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above-mentioned other operations and / or functions of each module / unit of the virtual object allocation device are to achieve Figure 3 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.
[0130] The present application also provides a computing device. Figure 5 As shown, this figure is a schematic diagram of a computing device provided in an embodiment of the present application, and the computing device 700 includes a bus 701, a processor 702, a communication interface 703 and a memory 704. The processor 702, the memory 704 and the communication interface 703 communicate through the bus 701.
[0131] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0132] The processor 702 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0133] The communication interface 703 is used for communicating with the outside.
[0134] The memory 704 may include a volatile memory, such as a random access memory (RAM). The memory 704 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0135] The memory 704 stores executable codes, and the processor 702 executes the executable codes to perform the aforementioned method for generating a panoramic image of a rope.
[0136] Specifically, in implementing Figure 4 In the case of the embodiment shown, and Figure 4 When each module or unit of the virtual object allocation device described in the embodiment is implemented by software, the execution Figure 4 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 704. The processor 702 executes the program code corresponding to each unit stored in the memory 704 to perform the aforementioned method for generating a panoramic image of a rope.
[0137] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned method for generating a panoramic image of a rope.
[0138] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.
[0139] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0140] When the computer program product is executed by a computer, the computer executes any of the aforementioned methods for generating a panoramic image of a rope. The computer program product may be a software installation package, and when any of the aforementioned methods for generating a panoramic image of a rope is required, the computer program product may be downloaded and executed on a computer.
[0141] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.
[0142] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.
Claims
1. A method for generating a panoramic image of a rope, characterized in that: The method comprises: Acquire multi-directional images of the rope to be tested; Segmenting the multi-directional image to obtain a rope image in the multi-directional image; According to the pre-fitted vertical coordinate transformation relationship and horizontal coordinate transformation relationship, the coordinates of each pixel in each rope image are corrected to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; According to the coordinates of each pixel in each rope image, determine the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image; Determine whether t is less than a direction number threshold, and obtain a first determination result, wherein the direction number threshold is greater than or equal to 3; If the first judgment result indicates that t is less than the direction number threshold, then according to the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the t-th direction, the first weight in the t-th direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1-th direction, and the second weight in the t+1-th direction, the pixel value of each pixel in the t-th overlapping area after the first to-be-overlapped area in the corrected rope image in the t-th direction overlaps with the second to-be-overlapped area in the corrected rope image in the t+1-th direction is determined; wherein t is a positive integer; If the first judgment result indicates that t is not less than the direction number threshold, the fused rope panoramic image is displayed according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinates of each pixel in the non-overlapping area.
2. The method according to claim 1, characterized in that: The coordinate transformation relationship in the horizontal direction is represented by the following formula: Among them, x″ i is the horizontal coordinate of the ith pixel after correction, x1 is the horizontal coordinate of the first pixel before correction, Δx n is the offset distance of the nth pixel in the horizontal direction, a1 is the pre-fitted first coefficient, b1 is the pre-fitted second coefficient, c1 is the pre-fitted third coefficient, x n is the horizontal coordinate of the nth pixel before correction, and X0 is the extreme point of the quadratic polynomial.
3. The method according to claim 1, characterized in that The coordinate transformation relationship in the vertical direction is represented by the following formula: Among them, y″ i is the ordinate of the ith pixel after correction, h is the height of the image, X0 is the extreme point of the quadratic polynomial, Y0 is the extreme value corresponding to the extreme point X0; a0 is the fourth coefficient of the pre-fitting, b0 is the fifth coefficient of the pre-fitting, and c0 is the sixth coefficient of the pre-fitting; x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
4. The method according to claim 1, characterized in that: The step of determining the pixel value of the corrected coordinate of each pixel in the corresponding corrected rope image according to the coordinate of each pixel in each rope image comprises: q(x″ i ,y″ i )=(q tr -q tl )x i +(q bl -q tl )y i +(q br +q tl -q bl -q tr )x i y i +q tl Among them, q(x″ i ,y″ i ) is the pixel value of the i-th pixel after correction, q tr is the pixel value of the pixel adjacent to the upper right of the i-th pixel before correction, q tl is the pixel value of the pixel adjacent to the upper left of the i-th pixel before correction, q bl is the pixel value of the pixel adjacent to the lower left of the i-th pixel before correction, q br is the pixel value of the pixel adjacent to the lower right of the i-th pixel before correction, x i is the horizontal coordinate of the i-th pixel before correction, y i is the ordinate of the i-th pixel before correction.
5. The method according to claim 1, characterized in that The method further comprises: receiving a marking result of the panoramic image of the rope from a user; If the marking result indicates that the health status score of the rope to be tested is lower than a score threshold, the position corresponding to the rope panoramic image in the rope to be tested is recorded.
6. The method according to claim 1, characterized in that The method further comprises: Performing color difference detection on the panoramic image of the rope to obtain a detection result; If the detection result indicates that there is a target area in the rope panoramic image whose color difference is greater than a preset color difference threshold, the target area is highlighted.
7. The method according to claim 1, characterized in that The step of acquiring multi-directional images of the rope to be tested comprises: At every preset distance, multi-directional images of the rope to be tested are obtained.
8. A device for generating a panoramic image of a rope, characterized in that: The device comprises: An acquisition module, used for acquiring multi-directional images of the rope to be tested; A segmentation module, used for segmenting the multi-directional image to obtain a rope image in the multi-directional image; A correction module, for correcting the coordinates of each pixel in each rope image according to the pre-fitted coordinate transformation relationship in the vertical direction and the coordinate transformation relationship in the horizontal direction, to obtain the corrected coordinates of each pixel in the corresponding corrected rope image; and determining the pixel value of the corrected coordinates of each pixel in the corresponding corrected rope image according to the coordinates of each pixel in each rope image; A judgment module, used to judge whether t is less than a direction number threshold, and obtain a first judgment result, wherein the direction number threshold is greater than or equal to 3; an iterative module, configured to determine, if the first judgment result representation t is less than a direction number threshold, the pixel value of each pixel in the tth overlapping area after the first to-be-overlapped area in the corrected rope image in the tth direction overlaps with the second to-be-overlapped area in the corrected rope image in the tth direction, according to the pixel value of each pixel in the first to-be-overlapped area in the corrected rope image in the tth direction, the first weight in the tth direction, the pixel value of each pixel in the second to-be-overlapped area in the corrected rope image in the t+1th direction, and the second weight in the t+1th direction; wherein t is a positive integer; A display module is used to display the fused rope panoramic image according to the pixel value of each pixel in the overlapping area of each corrected rope image and the pixel value of the corrected coordinate of each pixel in the non-overlapping area if the first judgment result representation t is not less than the direction number threshold.
9. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Steel wire rope detection device based on three-dimensional image, surface damage detection method and rope diameter calculation method
CN111721216A
Machine vision steel wire rope surface defect detection method based on multi-feature fusion
CN116843626A
Cableway surface image splicing method and device based on wavelet transformation and storage medium
CN117974438A
Rope inspection system, rope inspection method, and program
JP6542416B1
