A railway tunnel panoramic image synthesis monitoring method, device, equipment and medium

Abstract

The present application provides a method, device, equipment and medium for synthetic monitoring of a railway tunnel panorama. The method comprises the steps of: collecting multiple groups of initial image sequences for each section; fusing multiple initial images in each group to obtain a fused image; splicing multiple fused images to obtain an initial section image; post-processing the initial section image to obtain a target section image. The present application collects initial images of different angles and exposures to reduce the impact of the dim environment of the tunnel and improve the authenticity of the image. The initial images are fused to reduce the processing of redundant information and improve processing efficiency. When splicing the fused images, important edge structures are extracted, the extraction process is simplified, and the processing speed is improved. The structural deformation of the fused image is limited to ensure stability during splicing and avoid structural distortion. Through post-processing, the distortion and distortion of the image edges are eliminated, and the overall visual effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and railway tunnel safety technology, and in particular to a railway tunnel panoramic image synthesis monitoring method, device, equipment and medium. Background Art

[0002] The walls and tracks of train tunnels usually need to be inspected regularly to ensure the safe operation of trains. Traditional inspection methods rely on manual inspections, which are not only time-consuming but also highly dangerous.

[0003] To solve this problem, some existing systems have adopted image acquisition and online analysis methods for assistance. However, these existing image acquisition and stitching technologies still face the following challenges:

[0004] 1. The impact of low-brightness environment: The scene in the tunnel is dim, and the image acquisition effect is often poor. In addition, the lighting devices installed on the walls in the tunnel will also cause the problem of uneven light. Traditional image stitching algorithms are difficult to extract enough feature points for matching in low-brightness environments, thus affecting the overall stitching quality.

[0005] 2. Differences in camera shooting angles: Inspections require multiple cameras in fixed positions to shoot from different perspectives. Due to differences in camera angles and positions, as well as variable factors such as the curvature of the tunnel and the wall structure, there are large geometric differences in the images taken by the camera from different angles. This geometric difference makes it difficult for existing stitching algorithms to align accurately, resulting in obvious seams and distortion.

[0006] 3. Insufficient stitching algorithm accuracy: The tunnel environment is complex, including cracks and stains on the wall, and the joints of different materials. These subtle but important structures are easily overlooked or mismatched in image stitching, resulting in inaccurate stitching. Existing algorithms are often unable to cope with these subtle differences, especially when dealing with images with large perspective differences and complex backgrounds, and the accuracy of stitching is difficult to guarantee.

[0007] 4. Low efficiency and unsuitable for practical applications: Current traditional stitching algorithms only consider the algorithm itself and are not optimized for practical applications. Due to the lack of real data sets, current algorithms usually only use generative data sets for training and testing, which cannot meet the needs of certain specific tasks, such as tunnel detection. These specific tasks have higher requirements for the reliability, authenticity, and accuracy of the stitching results. On the other hand, the processing of high-resolution images requires a lot of computing resources, and the existing stitching algorithms also have significant deficiencies in computing efficiency. In addition, most of the current stitching algorithms can only stitch two pictures at a time, and when the number of cameras increases, the program needs to be run multiple times in succession. Therefore, especially when monitoring multiple platforms, multiple railways, and multiple tunnels at the same time, it is particularly important to optimize this problem because the server needs to frequently update data and run the stitching program. Summary of the invention

[0008] The present invention provides a method, device, equipment and medium for monitoring the synthesis of panoramic images of railway tunnels, so as to solve the problems that in the process of detecting railway tunnels, the existing image acquisition and splicing technology is affected by the low brightness environment of the tunnel and the difference in camera shooting angles, resulting in poor image splicing quality, obvious seams and distortion, low splicing accuracy, low efficiency and unapplicability.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is to provide a railway tunnel panoramic image synthesis monitoring method, comprising the steps of: dividing the railway tunnel into a plurality of continuous sections, respectively acquiring a plurality of groups of initial image sequences for each of the sections, each group of the initial image sequences comprising a plurality of initial images; wherein the shooting angles of any two groups of the initial image sequences are different, and the exposures of any two initial images in each group of the initial image sequences are different; fusing the plurality of initial images in each group to obtain a fused image corresponding to each group of the initial image sequences; splicing the plurality of fused images to obtain an initial section image corresponding to the section; and post-processing the initial section images to obtain a target section image.

[0010] In some embodiments, in the step of fusing the multiple initial images in each group, the fusion processing specifically includes: converting the multiple initial images from the RGB color space to the YCbCr color space; encoding and fusing the initial brightness data of the multiple initial images to obtain fused brightness data; using a multi-exposure fusion strategy, calculating the initial chromaticity data of the multiple initial images to obtain fused chromaticity data; performing weighted summation of the fused brightness data and the fused chromaticity data to obtain the fused image; and converting the fused image from the YCbCr color space back to the RGB color space.

[0011] In some embodiments, after the step of fusing the multiple initial images in each group to obtain the fused image corresponding to each group of the initial image sequence, the step further includes: uniformly cropping the multiple fused images according to a predetermined resolution, creating a stitching rule, and performing a rotation operation on the cropped fused images in sequence according to the stitching rule.

[0012] In some embodiments, in the step of stitching the multiple fused images, the stitching specifically includes: aligning the overlapping areas of the multiple fused images and determining the transformation relationship between the multiple fused images; creating a global similarity prior model based on the transformation relationship, and solving the transformation parameters of the global similarity prior model to obtain a transformation function; using the transformation function, respectively transforming the multiple fused images to obtain a plurality of corresponding transformed images; stitching the multiple transformed images to obtain the initial road section image.

[0013] In some embodiments, the transformation parameters include geometric structure parameters, weight parameters and similarity transformation parameters; the geometric structure parameters include edge feature parameters, straight line structure parameters and triangle construction parameters, the weight parameters include overlap parameters and non-overlap parameters, and the similarity transformation parameters include local similarity parameters and global similarity parameters.

[0014] In some embodiments, in the step of post-processing the initial road section image, the post-processing specifically includes: performing seam fusion on the spliced ​​parts within the initial road section image to obtain an optimized road section image; and performing neat cropping on the optimized road section image to obtain the target road section image.

[0015] In some embodiments, after the step of post-processing the initial road section image to obtain the target road section image, the step further includes: conducting an online review of the target road section image, and formulating a maintenance strategy if the review fails.

[0016] The present invention also provides a railway tunnel panoramic image synthesis monitoring device, comprising: an acquisition unit, used to divide the railway tunnel into a plurality of continuous sections, and respectively acquire a plurality of groups of initial image sequences of each section, each group of the initial image sequences including a plurality of initial images; wherein the shooting angles of any two groups of the initial image sequences are different, and the exposures of any two initial images in each group of the initial image sequences are different; a fusion unit, used to perform fusion processing on the plurality of the initial images in each group, and obtain a fused image corresponding to each group of the initial image sequences; a splicing unit, used to perform splicing processing on the plurality of the fused images, and obtain an initial section image corresponding to the section; and a post-processing unit, used to perform post-processing on the initial section images, and obtain a target section image.

[0017] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the computer program.

[0018] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0019] The beneficial effects of the present invention are as follows: the present application provides a method, device, equipment and medium for synthesizing a panoramic image of a railway tunnel. The method comprises the steps of: dividing the railway tunnel into a plurality of continuous sections, respectively acquiring a plurality of groups of initial image sequences of each section, each group of initial image sequences comprising a plurality of initial images; wherein the shooting angles of any two groups of initial image sequences are different, and the exposures of any two initial images in each group of initial image sequences are different; fusing the plurality of initial images in each group to obtain a fused image corresponding to each group of initial image sequences; splicing the plurality of fused images to obtain an initial section image corresponding to the section; post-processing the initial section image to obtain a target section image. By acquiring a plurality of initial images at different angles and exposures, the present application can reduce the influence of the dim environment of the tunnel on image acquisition, and effectively improve the quality and authenticity of the image. Further, by fusing the plurality of initial images in each group, not only the key information of each initial image is retained, but also the processing process of redundant information is reduced, saving the time required for the entire process, and effectively improving the processing efficiency of the present application for tunnel images. Furthermore, when the fused images are spliced, important edge structures in the fused images are extracted, which simplifies the feature extraction process, effectively reduces the complexity of the method of the present application, and further improves the processing speed. At the same time, the deformation of the structure of the fused image is restricted to ensure the stability of the structure during splicing and avoid structural distortion. Through post-processing, the initial road section image is optimized and cropped, which effectively eliminates the distortion and distortion of the image edge, making the image more natural and improving the overall visual effect. The image processing of railway tunnels in the present application is not only efficient and reliable, but also has strong practicality in complex scenes. The target section image finally obtained is not only of excellent quality, but also has further enhanced stability and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the process of a railway tunnel panoramic image synthesis monitoring method of the present application;

[0021] Figure 2 It is a data flow diagram of a railway tunnel panoramic image synthesis monitoring method of the present application;

[0022] Figure 3 It is a schematic diagram of an initial section image in a railway tunnel panoramic image synthesis monitoring method of the present application;

[0023] Figure 4 It is a schematic diagram of a target section image in a railway tunnel panoramic image synthesis monitoring method of the present application;

[0024] Figure 5 This is a schematic diagram of the composition of a railway tunnel panoramic image synthesis monitoring device of the present application;

[0025] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0026] Figure 7 It is a schematic block diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0027] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0028] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0029] like Figure 1 As shown, the present invention provides a railway tunnel panoramic image synthesis monitoring method, comprising the steps of:

[0030] S1: Divide the railway tunnel into a plurality of continuous sections, and collect a plurality of groups of initial image sequences for each section, each group of initial image sequences includes a plurality of initial images; wherein the shooting angles of any two groups of initial image sequences are different, and the exposures of any two initial images in each group of initial image sequences are different.

[0031] S2: performing fusion processing on multiple initial images in each group to obtain a fused image corresponding to each group of initial image sequences.

[0032] S3: stitching the multiple fused images to obtain an initial road section image corresponding to the road section.

[0033] S4: Post-process the initial road section image to obtain a target road section image.

[0034] By collecting multiple initial images with different exposures at different angles, the present application can reduce the impact of the dim environment of the tunnel on image acquisition, and effectively improve the quality and authenticity of the image. Further, by fusing multiple initial images in each group, not only the key information of each initial image is retained, but also the processing process of redundant information is reduced, saving the time required for the entire process, and effectively improving the processing efficiency of the tunnel image of the present application. Further, when splicing the fused images, the important edge structure in the fused images is extracted, the feature extraction process is simplified, the complexity of the method of the present application is effectively reduced, and the processing speed is further improved. At the same time, the deformation of the structure of the fused image is restricted to ensure the stability of the structure in splicing and avoid structural distortion. Through post-processing, the initial section image is optimized and cropped, and the distortion and distortion of the image edge are effectively eliminated, making the image more natural and improving the overall visual effect. The image processing of the railway tunnel in the present application is not only efficient and reliable, but also has strong practicality in complex scenes. The target section image finally obtained is not only of excellent quality, but also has further enhanced stability and reliability.

[0035] Combine the following Figures 1 to 7 , the present application is further described in detail with specific embodiments.

[0036] Figure 1 The following is a flow chart of a method for synthesizing a panoramic image of a railway tunnel provided by an embodiment of the present application, which is described in detail as follows:

[0037] S1: Divide the railway tunnel into a plurality of continuous sections, and collect a plurality of groups of initial image sequences for each section, each group of initial image sequences includes a plurality of initial images; wherein the shooting angles of any two groups of initial image sequences are different, and the exposures of any two initial images in each group of initial image sequences are different.

[0038] Specific, combined Figure 2 As shown, the railway tunnel 1 is divided into a plurality of continuous sections 11, and multi-eye cameras 12 are respectively arranged on the tunnel top wall on both sides of each section 11. Each multi-eye camera 12 includes a plurality of cameras, and the shooting setting angle of each camera is different. Through the multi-eye camera 12, a plurality of groups of initial image sequences (such as A1, A2, A3, ..., An) of each section 11 are respectively collected. Each group of initial image sequences includes a plurality of initial images 21; wherein, the shooting angles of any two groups of initial image sequences are different, and the exposure of any two initial images 21 in each group of initial image sequences are different.

[0039] For example, taking the initial image sequence A1 as an example, the initial image sequence A1 is a plurality of initial images 21 with different exposures taken by the same camera of the same multi-eye camera 12 (that is, the shooting angle of each initial image 21 is the same). The initial image sequence A1 and the initial image sequence A2 are two groups of images taken by two different cameras of the same multi-eye camera 12.

[0040] By collecting multiple images at different angles and exposures, the impact of the dim tunnel environment on image acquisition can be reduced, effectively improving the quality and authenticity of the image.

[0041] S2: performing fusion processing on multiple initial images in each group to obtain a fused image corresponding to each group of initial image sequences.

[0042] Specific, combined Figure 2 As shown, first, multiple initial images 21 in each group of initial image sequences are converted from the RGB color space to the YCbCr color space. It should be noted that the RGB color space is an additive color model based on the three basic colors of red (R), green (G), and blue (B). The RGB color space produces a wide range of color changes by adjusting the brightness of the three colors of red, green, and blue (represented by values ​​of 0 to 255, respectively). The YCbCr color space is a color space used for image and video processing, mainly used to improve compression efficiency and reduce bandwidth requirements, and consists of brightness data (Y), blue chrominance data (Cb), and red chrominance data (Cr). Converting the initial image 21 from the RGB color space to the YCbCr color space helps to avoid chromaticity distortion.

[0043] Furthermore, the initial brightness data of the multiple initial images 21 are coded and fused to obtain fused brightness data.

[0044] Specifically, N dictionary convolution units and one 1×1 convolution unit are used in sequence to encode the initial brightness data to obtain the first code. The first code is further converted into a weight map using an activation function, and a fusion operation is performed to obtain fused brightness data, which is expressed by the following formula:

[0045]

[0046] in, represents the fused brightness data, represents the weight graph, represents the initial brightness data, Representative Initial image 21, represents the total number of initial images 21 in a set of initial image sequences, Represents a fusion operation.

[0047] Furthermore, the initial chromaticity data (including initial blue chromaticity data and initial red chromaticity data) of the multiple initial images 21 are calculated using a multi-exposure fusion strategy to obtain fused chromaticity data (including fused blue chromaticity data and fused red chromaticity data), which is expressed by the following formula:

[0048] ;

[0049]

[0050] in, represents the fusion of blue chromaticity data, represents the initial blue chromaticity data, represents the fused red chromaticity data, Represents the initial red chromaticity data.

[0051] Further, a weighted sum is performed on the fused brightness data and the fused chrominance data to obtain a fused image 22. The fused image 22 is represented by the following formula:

[0052]

[0053] in, represents the fused image 22, represents the weight graph, Represents the weighted sum of fused luminance data and fused chrominance data.

[0054] Furthermore, the fused image 22 is converted from the YCbCr color space back to the RGB color space.

[0055] By fusing multiple initial images in each group, not only the key information of each initial image is retained, but also the processing of redundant information is reduced, saving the time required for the entire process, and effectively improving the processing efficiency of the tunnel image in this application.

[0056] Furthermore, in this embodiment, the quality of the obtained fused image 22 is evaluated using a similarity halo loss function, and the similarity halo loss function is expressed by the following formula:

[0057]

[0058] in, represents the similarity halo loss function, represents the prediction function, represents the similarity loss function, represents the penalty parameter, Represents the halo loss function.

[0059] Furthermore, in this embodiment, the obtained multiple fused images 22 are uniformly cropped according to a predetermined resolution, and a stitching rule is created. According to the stitching rule, the cropped fused images 22 are rotated in sequence.

[0060] Specifically, a predetermined resolution is first set. In the present application, the value of the predetermined resolution is set according to the track line area in each fused image 22. Cropping the multiple fused images 22 uniformly according to the predetermined resolution not only makes the multiple fused images 22 more uniform and standard, but also can crop out irrelevant areas in the fused images 22, thereby reducing the interference of irrelevant areas on image processing.

[0061] Next, a stitching rule is created. The stitching rule includes a stitching order and a stitching combination, and records the order of each fused image 22 in subsequent stitching, as well as its specific stitching object (ie, which fused images 22 are to be stitched together).

[0062] After determining the stitching sequence and stitching combination, all fused images 22 are classified and named for subsequent stitching processing. At the same time, according to the stitching combination, multiple fused images 22 in a combination are rotated to a uniform direction angle for subsequent direct stitching.

[0063] S3: stitching the multiple fused images to obtain an initial road section image corresponding to the road section.

[0064] In this embodiment, combined with Figure 2 As shown, the overlapping areas of the multiple fused images 22 are aligned, and the transformation relationship between the multiple fused images 22 is determined.

[0065] Specifically, multiple key points of each fused image 22 are extracted, and the key points of the overlapping areas of the multiple fused images 22 are matched to obtain multiple matching pairs. Then, the confidence of each matching pair is calculated using a ratio test function, and the matching pairs with high confidence are retained, and the matching pairs with low confidence are deleted to ensure the accuracy and reliability of the splicing order in the subsequent splicing. At the same time, the transformation relationship between the multiple fused images 22 is determined based on the multiple matching pairs.

[0066] Furthermore, according to the transformation relationship, a global similarity prior model is created, and the transformation parameters of the global similarity prior model are solved to obtain a conversion function. Among them, the transformation parameters include geometric structure parameters, weight parameters and similarity transformation parameters; the geometric structure parameters include edge feature parameters, straight line structure parameters and triangle construction parameters, the weight parameters include overlapping parameters and non-overlapping parameters, and the similarity transformation parameters include local similarity parameters and global similarity parameters.

[0067] In this embodiment, the conversion function is expressed by the following formula:

[0068]

[0069] in, represents the conversion function, represents the geometric structure parameters, represents the local similarity parameter, represents the global similarity parameter, represents the overlap parameter, Represents non-overlapping parameters.

[0070] Furthermore, the multiple fused images 22 are converted respectively using a conversion function to obtain multiple corresponding conversion images, and the multiple conversion images are spliced ​​to obtain an initial road section image 23 corresponding to the road section 11.

[0071] It should be noted that the method for solving the transformation parameters of the present application includes the steps of:

[0072] The large-scale edge features of each fused image 22 are extracted by an edge detection algorithm to obtain edge feature parameters. The straight line structure of each fused image 22 is extracted by a line segment detector to obtain straight line structure parameters. According to the large-scale edge features, triangle sampling is constructed to obtain triangle construction parameters. The edge feature parameters, straight line structure parameters and triangle construction parameters are combined to obtain geometric structure parameters.

[0073] Furthermore, according to the triangle construction parameters, constraints are set on the similarity of triangle sampling, the sampling point coordinates are calculated, and the local similarity parameters and the global similarity parameters are obtained.

[0074] Furthermore, the overlap parameter and the non-overlap parameter are determined according to the sampling point coordinates and the distance between the sampling point and the boundary of the overlap area.

[0075] S4: Post-process the initial road section image to obtain a target road section image.

[0076] Specific, combined Figure 2 As shown, post-processing specifically includes:

[0077] The spliced ​​parts in the initial road section image 23 are seam-fused to obtain an optimized road section image. In this embodiment, the seam fusion includes fade-in and fade-out and color smoothing operations. By seam-fusion of the spliced ​​parts, it can be ensured that the spliced ​​parts have a smooth transition and are displayed more naturally. Among them, the seam fusion method can adopt multi-band fusion technology.

[0078] Furthermore, the optimized road section image is trimmed to obtain the target road section image 24. Although the optimized road section image after seam fusion achieves seamless transition inside, the boundary is still irregular, which affects the appearance. Therefore, in this embodiment, the optimized road section image is further trimmed to effectively eliminate the distortion and distortion of the image edge, making the optimized road section image more natural, improving the overall visual effect, enhancing the appearance and observability, and facilitating subsequent manual inspection.

[0079] The regularization cropping specifically includes: determining the outermost non-empty pixel boundary of the optimized road segment image (ie, the minimum circumscribed matrix), performing an inward erosion operation on the optimized road segment image and cropping it according to the non-empty pixel boundary to form a standard rectangle, and obtaining the final target road segment image 24.

[0080] Combination Figure 3 , Figure 4 As shown, the initial section image 23 and the target section image 24 corresponding to one section 11 of the railway tunnel 1 of the present application are shown. It can be seen that the final target section image 24 is not only clear, with a clear view of the corresponding section 11, but also has no splicing marks and excellent quality.

[0081] Further, after obtaining the target road section image 24, the target road section image 24 is uploaded. The target road section image 24 is reviewed online. If the review is unqualified, a corresponding maintenance strategy is formulated, and relevant staff are arranged to inspect and repair the railway tunnel 1 in time, effectively ensuring the driving safety of the line. If the review is qualified, the collection and monitoring will continue.

[0082] Corresponding to the railway tunnel panoramic image synthesis monitoring method of the above embodiment, Figure 5 A schematic diagram of the composition of a railway tunnel panoramic image synthesis monitoring device provided in an embodiment of the present application is shown. For the sake of ease of explanation, only the parts related to the embodiment of the present application are shown.

[0083] refer to Figure 5 The railway tunnel panoramic image synthesis monitoring device comprises:

[0084] The acquisition unit 101 is used to divide the railway tunnel into a plurality of continuous sections, and respectively acquire a plurality of groups of initial image sequences of each section, each group of initial image sequences including a plurality of initial images; wherein the shooting angles of any two groups of initial image sequences are different, and the exposures of any two initial images in each group of initial image sequences are different.

[0085] The fusion unit 102 is used to perform fusion processing on multiple initial images in each group to obtain a fused image corresponding to each group of initial image sequences.

[0086] The stitching unit 103 is used to stitch the multiple fused images to obtain an initial road section image corresponding to the road section.

[0087] The post-processing unit 104 is used to post-process the initial road section image to obtain a target road section image.

[0088] It should be noted that since the above-mentioned railway tunnel panoramic image synthesis monitoring device is based on the same concept as the embodiment of the method of the present application, its specific functions, the process of each module realizing its respective function and the technical effects brought about can be specifically referred to the description of the embodiment shown in the above-mentioned railway tunnel panoramic image synthesis monitoring method, and will not be repeated here.

[0089] Based on the same inventive concept, the present application also provides an electronic device, which includes a processor, a memory and a communication circuit, wherein the processor is connected to the memory and the communication circuit respectively; wherein the communication circuit is used for communication connection, the memory is used to store a computer program, and the processor is used to execute the computer program to realize the above-mentioned railway tunnel panoramic image synthesis monitoring method.

[0090] See also Figure 6 The electronic device described in the embodiment of the present application may specifically include a processor 210 and a memory 220. The memory 220 is coupled to the processor 210.

[0091] The processor 210 is used to control the operation of the electronic device. The processor 210 may also be referred to as a CPU (Central Processing Unit). The processor 210 may be an integrated circuit chip with signal processing capabilities. The processor 210 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0092] The memory 220 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 220 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 220 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 220 is used to store at least one program code.

[0093] The processor 210 is used to execute the computer program stored in the memory 220 to implement the methods described in the various method embodiments of the present application.

[0094] In some embodiments, the electronic device may further include: a peripheral terminal interface 230 and at least one peripheral terminal. The processor 210, the memory 220 and the peripheral terminal interface 230 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 230 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 240, a display screen 250, an audio circuit 260 and a power supply 270.

[0095] The peripheral terminal interface 230 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 210 and the memory 220. In some embodiments, the processor 210, the memory 220, and the peripheral terminal interface 230 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 210, the memory 220, and the peripheral terminal interface 230 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0096] The radio frequency circuit 240 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 240 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 240 is the communication circuit of the electronic device. The radio frequency circuit 240 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 240 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 240 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 240 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0097] The display screen 250 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 250 is a touch display screen, the display screen 250 also has the ability to collect touch signals on the surface or above the surface of the display screen 250. The touch signal can be input to the processor 210 as a control signal for processing. At this time, the display screen 250 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 250 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 250 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 250 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 250 can also be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 250 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0098] The audio circuit 260 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 210 for processing, or input them into the radio frequency circuit 240 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 210 or the radio frequency circuit 240 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 260 may also include a headphone jack.

[0099] The power supply 270 is used to power various components in the electronic device. The power supply 270 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 270 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0100] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0101] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] Based on the same inventive concept, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above-mentioned railway tunnel panoramic image synthesis monitoring method.

[0105] See also Figure 7 , if the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 300. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage media.

[0106] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, and will not be repeated here.

[0107] It can be seen that the present invention discloses a method, device, equipment and medium for synthesizing a panoramic image of a railway tunnel. The method includes the steps of: dividing the railway tunnel into a plurality of continuous sections, respectively acquiring a plurality of groups of initial image sequences of each section, each group of initial image sequences including a plurality of initial images; wherein the shooting angles of any two groups of initial image sequences are different, and the exposures of any two initial images in each group of initial image sequences are different; fusing the plurality of initial images in each group to obtain a fused image corresponding to each group of initial image sequences; splicing the plurality of fused images to obtain an initial section image corresponding to the section; post-processing the initial section image to obtain a target section image. By acquiring a plurality of initial images at different angles and exposures, the present application can reduce the influence of the dim environment of the tunnel on image acquisition, and effectively improve the quality and authenticity of the image. Further, by fusing the plurality of initial images in each group, not only the key information of each initial image is retained, but also the processing process of redundant information is reduced, saving the time required for the entire process, and effectively improving the processing efficiency of the present application for tunnel images. Furthermore, when the fused images are spliced, important edge structures in the fused images are extracted, which simplifies the feature extraction process, effectively reduces the complexity of the method of the present application, and further improves the processing speed. At the same time, the deformation of the structure of the fused image is restricted to ensure the stability of the structure during splicing and avoid structural distortion. Through post-processing, the initial road section image is optimized and cropped, which effectively eliminates the distortion and distortion of the image edge, making the image more natural and improving the overall visual effect. The image processing of railway tunnels in the present application is not only efficient and reliable, but also has strong practicality in complex scenes. The target section image finally obtained is not only of excellent quality, but also has further enhanced stability and reliability.

[0108] The above are only embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A railway tunnel panoramic image synthesis monitoring method, characterized in that: Includes steps: Divide the railway tunnel into a plurality of continuous sections, and respectively acquire a plurality of groups of initial image sequences of each section, wherein each group of the initial image sequences includes a plurality of initial images; wherein the shooting angles of any two groups of the initial image sequences are different, and the exposure degrees of any two initial images in each group of the initial image sequences are different; Performing fusion processing on the multiple initial images in each group to obtain a fused image corresponding to each group of the initial image sequence; Performing splicing processing on the multiple fused images to obtain an initial road section image corresponding to the road section; Post-processing the initial road section image to obtain a target road section image; Wherein, in the step of performing stitching processing on the plurality of fused images, the stitching processing specifically includes: Aligning overlapping areas of the multiple fused images and determining a transformation relationship between the multiple fused images; According to the transformation relationship, a global similarity prior model is created, and the transformation parameters of the global similarity prior model are solved to obtain a conversion function; Using the conversion function, converting the plurality of fused images respectively to obtain a plurality of corresponding converted images; The multiple converted images are spliced ​​together to obtain the initial road section image.

2. The railway tunnel panoramic image synthesis monitoring method according to claim 1 is characterized in that: In the step of performing fusion processing on the multiple initial images in each group, the fusion processing specifically includes: Converting the plurality of initial images from the RGB color space to the YCbCr color space; Performing encoding and fusion processing on the initial brightness data of the plurality of initial images to obtain fused brightness data; Calculating the initial chromaticity data of the plurality of initial images by using a multi-exposure fusion strategy to obtain fused chromaticity data; Performing weighted summation on the fused luminance data and the fused chrominance data to obtain the fused image; The fused image is converted from the YCbCr color space back to the RGB color space.

3. The railway tunnel panoramic image synthesis monitoring method according to claim 1 is characterized in that: After the step of fusing the multiple initial images in each group to obtain the fused image corresponding to each group of the initial image sequence, the step further includes: The plurality of fused images are uniformly cropped according to a predetermined resolution respectively, and a splicing rule is created. According to the splicing rule, a rotation operation is sequentially performed on the cropped fused images.

4. The railway tunnel panoramic image synthesis monitoring method according to claim 1, characterized in that: The transformation parameters include geometric structure parameters, weight parameters and similar transformation parameters; the geometric structure parameters include edge feature parameters, straight line structure parameters and triangle construction parameters, the weight parameters include overlapping parameters and non-overlapping parameters, and the similar transformation parameters include local similarity parameters and global similarity parameters.

5. The railway tunnel panoramic image synthesis monitoring method according to claim 1, characterized in that: In the step of post-processing the initial road section image, the post-processing specifically includes: Performing seam fusion on the spliced ​​parts in the initial road section image to obtain an optimized road section image; The optimized road section image is cropped to obtain the target road section image.

6. The railway tunnel panoramic image synthesis monitoring method according to claim 1, characterized in that: After the step of post-processing the initial road section image to obtain the target road section image, the step further includes: The target road section image is reviewed online, and if the review fails, a maintenance strategy is formulated.

7. A railway tunnel panoramic image synthesis monitoring device, characterized in that: include: A collection unit is used to divide the railway tunnel into a plurality of continuous sections, and respectively collect a plurality of groups of initial image sequences of each section, each group of the initial image sequences comprising a plurality of initial images; wherein the shooting angles of any two groups of the initial image sequences are different, and the exposure degrees of any two initial images in each group of the initial image sequences are different; A fusion unit, used for fusing the multiple initial images in each group to obtain a fused image corresponding to each group of the initial image sequence; A stitching unit, used for stitching the multiple fused images to obtain an initial road section image corresponding to the road section; wherein the stitching process specifically includes: aligning the overlapping areas of the multiple fused images and determining the transformation relationship between the multiple fused images; creating a global similarity prior model according to the transformation relationship, and solving the transformation parameters of the global similarity prior model to obtain a conversion function; using the conversion function, respectively transforming the multiple fused images to obtain a plurality of corresponding conversion images; stitching the multiple conversion images to obtain the initial road section image; The post-processing unit is used to post-process the initial road section image to obtain a target road section image.

8. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images