Rail visual perception method and device using binocular multiple exposure

Abstract

The application discloses a kind of binocular multiple exposure steel rail visual perception method and device, it is related to steel rail image processing technical field, the method includes: obtaining the low exposure original graph of binocular camera shooting steel rail, high exposure original graph;It is divided into multiple blocks in low exposure original graph, high exposure original graph respectively, the block feature of each block is calculated;The block feature includes local energy, local quality and spatial consistency;Local energy reflects the proportion of effective signal related to laser stripe, local quality reflects the similarity between multiple sub-regions based on block division, and spatial consistency reflects the vertical distance of block and light band energy center;Low exposure initial weight graph and high exposure initial weight graph are obtained by using the block feature calculated weighted fusion;Low exposure initial weight graph, high exposure initial weight graph are guided filtering weight correction, and modified weight graph is used to carry out weighted fusion pixel by pixel to obtain fusion image.The application can retain significant features and spatial details in image, improve steel rail image fusion quality.

Description

Technical Field

[0001] The present invention relates to the technical field of rail image processing, and in particular to a binocular multiple-exposure rail visual perception method and device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] At present, track inspection mainly adopts dynamic non-contact methods, such as high-speed comprehensive inspection trains, track inspection vehicles and rail flaw detection vehicles, among which structured light technology is one of the important components. The rail imaging system based on structured light principle faces a series of technical challenges when collecting images under natural light. Imaging principle, such as Figure 1 As shown in Figure 2, two cameras and two lasers are used to capture images of the rails. However, the dynamic range of natural light intensity is very wide, ranging from as low as 100 at night. cd / , accessible during the day cd / However, the dynamic range that ordinary cameras can capture is only 0-255, and the image information obtained cannot fully represent the high dynamic range image (HDR) of natural scenes. In addition, due to the limitations of hardware equipment technology and external environmental influences such as ambient light, weather and sun altitude, overexposure and underexposure often occur, resulting in the image captured by a single sensor being unable to effectively and comprehensively describe the content of the imaging scene. Therefore, reference Figure 1 ,The currently collected rail images have the following problems:

[0004] (1) Ambient light interference: Strong light in natural light can interfere with the clarity of the structured light projection pattern, causing distortion of the encoded information received by the camera. This interference will directly reduce the measurement accuracy and even make some areas unable to be decoded;

[0005] (2) Insufficient dynamic range: Natural scenes contain both bright and shadow areas. A single exposure setting makes it difficult to capture details in both bright and dark areas. For example, bright areas may be overexposed, while shadow areas may be underexposed, resulting in information loss.

[0006] (3) Differences in surface reflection characteristics: Objects of different materials have different surface reflectivities. High-reflectivity surfaces (such as the light strip area of ​​the rail) may cause overexposure of the structured light pattern, while low-reflectivity surfaces have weak signals that are difficult to decode.

[0007] To address these issues, existing approaches typically employ two methods to improve the dynamic range of imaging devices: hardware design and software techniques. The former can directly improve exposure efficiency and image quality, but is costly and has limited practicality. Software techniques employ a camera response function (CRF) to reconstruct high dynamic range images. Multi-exposure image fusion (MEF) fuses input images of varying exposures into an information-rich, vibrant image without requiring camera curve calibration, HDR reconstruction, or tone mapping. Compared to the first approach, MEF offers a simpler, more economical, and more efficient approach. However, software techniques are typically more general solutions, generating a weight map for each input image and calculating the fused image as the weighted average of all input images. Existing methods are not well suited for fusion processing to improve the dynamic range of rail images. They lose significant features and useful information, resulting in a decrease in the quality of the fused rail images and an inability to meet subsequent image processing requirements. Summary of the Invention

[0008] An embodiment of the present invention provides a binocular multiple-exposure rail visual perception method for preserving significant features and spatial details in an image and improving the quality of rail image fusion. The method includes:

[0009] Obtain low-exposure and high-exposure original images of the same rail position taken by the binocular camera and laser;

[0010] The low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated. The block features include local energy, local quality, and spatial consistency. Local energy reflects the proportion of effective signals related to the laser stripe, local quality reflects the similarity between multiple sub-regions based on the block division, and spatial consistency reflects the vertical distance between the block and the energy center of the light stripe.

[0011] The low-exposure initial weight map is obtained by weighting and fusion using the block features of each block of the low-exposure original image. The high-exposure initial weight map is obtained by weighting and fusion using the block features of each block of the high-exposure original image.

[0012] Based on the guided filtering principle, the low-exposure original image is used as a guide to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map, respectively, to obtain the low-exposure weight correction map and the high-exposure weight correction map;

[0013] The low-exposure weight correction map and the high-exposure weight correction map are used to perform weighted fusion pixel by pixel to obtain the fused image.

[0014] An embodiment of the present invention further provides a binocular multiple-exposure rail visual perception device for preserving significant features and spatial details in an image and improving the quality of rail image fusion. The device comprises:

[0015] Image data acquisition module, used to obtain low-exposure original images and high-exposure original images of the same position on the rail taken by the binocular camera and laser;

[0016] A block feature calculation module is used to divide the low-exposure original image and the high-exposure original image into multiple blocks and calculate the block features of each block; the block features include local energy, local quality, and spatial consistency; local energy reflects the proportion of effective signals related to the laser stripe, local quality reflects the similarity between multiple sub-regions based on the block division, and spatial consistency reflects the vertical distance between the block and the energy center of the light stripe;

[0017] The initial weight map calculation module is used to use the block features of each block of the low-exposure original image to perform weight distribution and weighted fusion to obtain a low-exposure initial weight map, and to use the block features of each block of the high-exposure original image to perform weight distribution and weighted fusion to obtain a high-exposure initial weight map;

[0018] The weight correction module is used to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map based on the guided filtering principle, and obtain the low-exposure weight correction map and the high-exposure weight correction map respectively;

[0019] The fusion processing module is used to perform weighted fusion pixel by pixel using the low-exposure weight correction map and the high-exposure weight correction map to obtain a fused image.

[0020] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned binocular multiple-exposure rail visual perception method is implemented.

[0021] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned binocular multiple-exposure rail visual perception method.

[0022] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned binocular multiple exposure rail visual perception method.

[0023] In the embodiment of the present invention, a low-exposure original image and a high-exposure original image are collected, and the low-exposure original image and the high-exposure original image are divided into blocks respectively. Then, an initial weight map is calculated, that is, the local energy, local quality and spatial consistency feature distribution of each block are integrated. Finally, a modified weight map is calculated based on guided filtering, and a fused image is obtained based on the weighted sum of each pixel. The embodiment of the present invention fully considers the characteristics of the structured light image of the rail, proposes a similarity measurement for further dividing the blocks into regions, and utilizes the characteristics of the rail light band being centered and the gauge angle having a fixed vertical position, proposes spatial consistency guided by the center of the light band energy, retains the information of the core detection area of ​​the rail head, directly filters out invalid information in the non-rail head area, and strengthens the quality assessment of the key area of ​​the rail head. The embodiment of the present invention effectively solves the problem of rail image quality degradation caused by ambient light interference, insufficient dynamic range and differences in surface reflection characteristics. Compared with the existing technology, the embodiment of the present invention better retains significant objects and spatial details in the image, integrates the image details of the rail affected by surface reflection, and improves the fusion quality of the rail image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 This is a schematic diagram of the structured light rail imaging principle in the prior art;

[0026] Figure 2 Schematic diagram of the flow of the rail visual perception method with binocular multiple exposures in an embodiment of the present invention;

[0027] Figure 3 This is a diagram showing a specific example of a method for visually perceiving rails using binocular multiple exposures according to an embodiment of the present invention;

[0028] Figure 4 FIG2 is another specific example of the rail visual perception method using binocular multiple exposures according to an embodiment of the present invention;

[0029] Figure 5 Schematic diagram of spatial consistency calculation in an embodiment of the present invention;

[0030] Figure 6 Schematic diagram of a low-exposure original image and a high-exposure original image in an embodiment of the present invention;

[0031] Figure 7 Schematic diagram comparing the fusion results of different algorithms in an embodiment of the present invention;

[0032] Figure 8 Schematic diagram of a rail visual perception device with binocular multiple exposures in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0034] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.

[0035] At present, track inspection mainly adopts dynamic non-contact methods, such as high-speed comprehensive inspection trains, track inspection vehicles and rail flaw detection vehicles, among which structured light technology is one of the important components. The rail imaging system based on structured light principle faces a series of technical challenges when collecting images under natural light. Imaging principle, such as Figure 1 As shown. These include (1) ambient light interference: strong light in natural light will interfere with the clarity of the structured light projection pattern, causing distortion of the encoded information received by the camera. This interference will directly reduce the measurement accuracy and even make some areas undecodeable; (2) insufficient dynamic range: there are highlights and shadows in natural scenes, and a single exposure setting is difficult to capture the details of both bright and dark areas at the same time. For example, the bright area may be overexposed, while the shadow area may be underexposed, resulting in information loss; (3) differences in surface reflection characteristics: objects of different materials have different surface reflectivity. High-reflectivity surfaces (such as the light strip area of ​​the rail) may cause the structured light pattern to be overexposed, while the low-reflectivity surface signal is weak and difficult to decode. Multi-exposure image fusion technology provides an effective way to solve these problems.

[0036] Typical image fusion methods can be categorized as spatial-domain, transform-domain, and deep learning-based. Spatial-domain methods fuse images directly in the spatial domain based on their spatial features. The general process involves generating a weight map for each input image and computing the fused image as the weighted average of all input images. Transform-domain methods typically consist of three stages: image conversion, coefficient fusion, and inverse transformation. They are generally categorized as multi-scale decomposition-based, gradient-domain-based, and sparse representation-based. In recent years, deep learning techniques have achieved tremendous success in many computer vision fields, such as object tracking and detection, due to their powerful representation capabilities. However, they are computationally complex, require high-quality equipment, and exhibit poor stability. These techniques include convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and feedforward neural networks (FNNs). Based on the model used, image fusion methods can be divided into supervised and unsupervised approaches. However, existing fusion methods are mostly applied to natural scenes with rich RGB images and do not consider the feature distribution of multi-exposure image sequences. Therefore, the existing methods cannot be well applied to the fusion processing of rail images with improved dynamic range. The rail images fused using the existing methods are not natural, have low quality, and even produce artifacts or blur.

[0037] In order to solve the defects in the existing rail image fusion processing technology, the embodiment of the present invention proposes a binocular multiple exposure rail visual perception method. Figure 2 FIG. 1 is a flow chart of a method for visually perceiving rails using binocular multiple exposures according to an embodiment of the present invention. Figure 2 As shown, the method includes:

[0038] Step 201: Obtain a low-exposure original image and a high-exposure original image of the same position of the rail taken by a binocular camera and a laser;

[0039] Step 202: Divide the low-exposure original image and the high-exposure original image into multiple blocks, and calculate the block features of each block; the block features include local energy, local quality, and spatial consistency; local energy reflects the proportion of effective signals related to the laser stripe, local quality reflects the similarity between multiple sub-regions based on the block division, and spatial consistency reflects the vertical distance between the block and the energy center of the light stripe;

[0040] Step 203: weighting and weighted fusion are performed using the block features of each block of the low-exposure original image to obtain a low-exposure initial weight map; weighting and weighted fusion are performed using the block features of each block of the high-exposure original image to obtain a high-exposure initial weight map;

[0041] Step 204: Based on the guided filtering principle, the low-exposure original image is used as a guide to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map, respectively, to obtain a low-exposure weight correction map and a high-exposure weight correction map;

[0042] Step 205: Perform weighted fusion pixel by pixel using the low-exposure weight correction map and the high-exposure weight correction map to obtain a fused image.

[0043] from Figure 2 As can be seen from the illustrated process, in this embodiment of the present invention, two multi-exposure image sequences are first acquired using a binocular camera to generate high- and low-exposure images. The rail region of interest (rail head) is then divided into patches (blocks). Next, an initial weight map is calculated, integrating the local energy, local mass, and spatial consistency feature distributions of each patch. Finally, a modified weight map is calculated using guided filtering, and a weighted summation of each pixel is performed to produce the fused image. Compared with existing methods, this algorithm better preserves salient objects and spatial details in the image and can also incorporate image details of the rail affected by surface reflections. While maintaining image quality, it reduces reliance on complex processing, demonstrating its significant practical application value.

[0044] In step 201, a low-exposure original image and a high-exposure original image of the same position of the rail taken by a binocular camera and a laser are obtained.

[0045] For example, in a binocular camera, one camera takes a high-exposure image, and the other camera takes a low-exposure image.

[0046] To further improve the quality of the original image, in one embodiment, obtaining a low-exposure original image and a high-exposure original image of the same position of the rail taken by a binocular camera and a laser may include:

[0047] Acquire multiple exposure images of the same rail position taken by the binocular camera and laser. The multiple exposure images include a low-exposure image from the left camera, a high-exposure image from the left camera, a low-exposure image from the right camera, and a high-exposure image from the right camera.

[0048] Using the preset quality assessment method, images with higher quality scores are screened out from the low-exposure images of the left camera and the low-exposure images of the right camera and recorded as low-exposure original images, and images with higher quality scores are screened out from the high-exposure images of the left camera and the high-exposure images of the right camera and recorded as high-exposure original images.

[0049] Figure 3 FIG. 1 is another specific example of the rail visual perception method using binocular multiple exposures according to an embodiment of the present invention. Figure 3 The overall process of rail structured light image fusion processing is demonstrated. Figure 3 In the figure, the left camera obtains a set of high- and low-exposure images, and the right camera obtains a set of high- and low-exposure images. For example, a chip specially designed for camera imaging is used. During imaging, the camera chip performs high exposure and low exposure alternately in the order of rows, obtaining two images in one exposure, one high exposure and one low exposure.

[0050] In a preferred embodiment, the preset quality assessment method includes: calculating an image entropy value, and treating an image with a larger image entropy value as an image with a higher quality score.

[0051] In this embodiment, image entropy is used as a screening metric, selecting images with higher entropy values ​​as high-quality images. Image entropy reflects the degree of disorder (information content) in the pixel grayscale distribution. Higher entropy values ​​indicate richer details and textures in the image, making it suitable for preserving details in the rail head area (such as the gauge angle and light band edge) during rail inspection. Furthermore, image entropy calculation is simple, requiring only grayscale histogram statistics and no manual intervention. It can quickly compare the information richness of binocular camera images with the same exposure (e.g., a low-exposure image from the left camera and a low-exposure image from the right camera), meeting real-time screening requirements.

[0052] Those skilled in the art may also utilize methods such as clarity analysis and noise level measurement when screening images.

[0053] refer to Figure 3 In , before calculating the block features of each block, image registration can be performed to match the positions of the filtered low-exposure original image and the high-exposure original image to facilitate subsequent calculations.

[0054] In step 202, the low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated, for example Figure 3 In the figure, the low-exposure original image and the high-exposure original image are divided into multiple blocks respectively.

[0055] The local energy reflects the effective signal ratio associated with the laser stripes. In one embodiment, the low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated, which may include:

[0056] The local energy of each block is calculated as follows:

[0057] The first threshold is used to distinguish valid signals from invalid areas, and the valid signal ratio of each block is calculated.

[0058] In the embodiment of the present invention, local energy is an energy screening mechanism adapted to the reflective characteristics of the rail surface. In the prior art, local energy calculation mostly uses general gradient or grayscale variance. In the embodiment of the present invention, a special design is made for the binary characteristics of high reflectivity of the light band on the rail surface and low reflectivity at the gauge angle.

[0059] This embodiment of the present invention uses a first threshold for pixel intensity to screen valid signal areas, calculate the regional energy percentage, and suppress noise and background interference. The first threshold can be set based on the pixel mean of each block, and is used to distinguish valid signals (such as structured light streaks on the rail surface) from invalid areas (such as noise or overexposed or underexposed areas). The higher the valid signal percentage and the more concentrated the energy distribution (constrained by the exponential term), the greater the weight of the block, ensuring that high-energy (clear and valid) areas dominate the fusion process. Blocks with high noise or primarily background content generally have lower energy values ​​and a smaller weight.

[0060] In one embodiment, using the first threshold to distinguish between valid signals and invalid areas and calculating the valid signal ratio of each block may include: using the first threshold to distinguish between valid signals and invalid areas and calculating the valid signal ratio of each block according to the following formula:

[0061] (1)

[0062] Where, E i represents the local energy value of the i-th block, α and β are coefficients, μ i is the grayscale mean of all pixels in the i-th block, p i 、ratio i , γ i (x, y) is the intermediate calculation amount, I i (x, y) represents the grayscale of the pixel (x, y) in the i-th block, N=2.

[0063] The local energy calculated by the above formula (1) has the following characteristics:

[0064] (1) Overexplosion suppression in the rail light band area: βμ i is a dynamic threshold, β is for example 0.8, based on the block pixel mean μ i Adaptive adjustment, only retaining the brightness higher than the block mean μ i 80% of the pixels (corresponding to the strong reflection signal in the rail light band area), filtering out the low reflection background (such as the rail bottom and roadbed) and underexposed noise. i Indicates the effective signal pixel ratio. When the light band area is overexposed due to strong light and the pixels are saturated, it can still be detected by high ratio. i Assigning high weights can prevent saturated pixels from being misjudged as invalid signals in existing traditional methods.

[0065] (2) Enhancement of low reflection area at gauge angle: exp(-α(p i -βμ i )) 2 Constraint effective energy p i Towards the threshold βμ i Concentrated, low reflection area such as gauge angle (pi Close to βμ i ) is given a medium weight to avoid being ignored due to weak signals, effectively solving the problem of detail loss in low-reflection areas in existing technologies.

[0066] (3) Noise block suppression: When there is random noise in the block, the ratio i Drop, E i It approaches 0, effectively eliminating discrete noise interference, while the general energy model in the existing technology is prone to misjudge noise as high-energy details.

[0067] Local quality reflects the similarity between multiple sub-regions based on block division. The global structural similarity in existing technologies cannot distinguish the quality differences between different areas of the rail head. In this embodiment of the present invention, a regional structural similarity metric is proposed to address the mixed characteristics of the smooth and reflective light band on the rail surface and the complex texture of the rail head. The sliding window is used to disassemble the intra-block structure to enhance the quality assessment of key areas of the rail head.

[0068] In one embodiment, dividing the low-exposure original image and the high-exposure original image into a plurality of blocks and calculating the block feature of each block may include calculating the local quality of each block as follows:

[0069] Each block is divided into multiple sub-regions, and the SSIM (Structural Similarity Index) values ​​between the sub-regions are calculated.

[0070] Structural similarity is used to evaluate the structural consistency of pixels within a block, preserving areas with rich detail. In implementation, the image block is divided into multiple sub-regions, and the structural similarity between them is calculated. Higher similarity indicates more regular structure within the block (e.g., the uniformly reflective area of ​​a railroad light strip) and better quality. Figure 4 FIG is another specific example of the rail visual perception method of binocular multiple exposure in an embodiment of the present invention, referring to FIG. Figure 4 ,shows (a) valid block of rail light band and (b) noise block.,Local quality reflection is used to distinguish valid signal blocks (such as rail texture),from noise blocks (such as random light spots) to avoid noise being incorrectly fused.

[0071] During implementation, each block can be divided into multiple sub-regions according to the following formula, and the structural similarity SSIM value between the sub-regions can be calculated:

[0072] (2)

[0073] Where Q i represents the local quality value of the i-th block, Slide represents the number of sub-regions divided by the block, S(u,v) represents the structural similarity measure between the u-th sub-region and the v-th sub-region, represents the variance of the u-th sub-region, represents the variance of the vth sub-region, δ uv represents the covariance between the u-th sub-region and the v-th sub-region, and C1 is a preset constant.

[0074] The local mass is calculated by the above formula (2), which has the following characteristics:

[0075] (1) Rail head region segmentation strategy: Each image block (e.g., 16×16 pixels) is divided into three equidistant sub-regions (Slide=3) along the longitudinal direction of the rail head, corresponding to the rail top surface (light band), gauge angle (detection key area), and rail head side (transition area), respectively. Figure 4 As shown. Light band area ( Figure 4 (a) in the middle is close to 1 (high gradient consistency), and is given a high weight; the noise block ( Figure 4 The fluctuation of (b) is large and the quality score is low, which solves the problem in the existing technology that the light band area is mistakenly judged as low quality due to the smooth transition.

[0076] (2) Anti-reflection interference optimization: For gradient saturation caused by high reflection in the light band area, the quality is evaluated by weighting the similarity between sub-regions (rather than a single gradient amplitude), avoiding the defect of high-reflection areas being incorrectly fused due to gradient disappearance in existing technologies.

[0077] Spatial consistency reflects the vertical distance between the block and the energy center of the light band. In an embodiment of the present invention, a vertical positioning method guided by the energy center of the light band is proposed. The geometric prior of the center of the rail light band and the fixed vertical position of the gauge angle is used to forcibly retain the core detection area information of the rail head.

[0078] Figure 5 This is a schematic diagram of spatial consistency calculation in an embodiment of the present invention, combined with Figure 5 The position of each block in the image and the position of the energy center of the light band can be calculated as follows:

[0079] (3)

[0080] Where, SC i represents the spatial consistency value of the i-th block, is the vertical position of the ith block, and H is the vertical position of the light band energy center.

[0081] In this example, the weighting of key rail head regions is emphasized, based on the energy center of the light band. The vertical distance between the image block and the energy center of the light band is calculated. A closer distance indicates proximity to the core detection area of ​​the rail head (e.g., the gauge angle), resulting in a higher weighting. This ensures that critical details such as the rail top surface and gauge angle are prioritized during fusion, while suppressing interference from irrelevant background elements (e.g., the trackbed and fasteners) away from the light band.

[0082] The spatial consistency calculated by the above formula (3) has the following characteristics:

[0083] (1) Dynamic positioning of the light strip energy center: H defines the ROI by using the rail head width prior (the rail head width corresponding to a 1435mm gauge is approximately 70mm), and calculates the weighted mean of the vertical coordinate y of the pixels in this area (the weight is the pixel intensity), automatically adapting to the vertical offset of the light strip due to wear and lighting changes. Existing technologies rely on fixed templates to match light strips and are unable to adapt to the vertical differences of different rail types (such as 60kg / m and 75kg / m). In the embodiments of the present invention, dynamic H is used to achieve model adaptation.

[0084] (2) Gauge angle area protection: The gauge angle is located about 15mm vertically below the light band, and its SC i When it is close to 1, the fusion weight is higher than that of the background area, ensuring that the details of the track angle edge lost due to underexposure in the low-exposure image are supplemented by the corresponding blocks in the high-exposure image.

[0085] (3) Anti-track bed background interference: S in the track bed area away from the light band i It approaches 0, directly filtering out invalid information in the non-rail head area. However, the global consistency model in the existing technology is easily disturbed by the roadbed texture, resulting in the fused image containing redundant background noise.

[0086] In step 203, the block features of each block of the low-exposure original image are used to perform weight distribution and weighted fusion to obtain a low-exposure initial weight map, and the block features of each block of the high-exposure original image are used to perform weight distribution and weighted fusion to obtain a high-exposure initial weight map.

[0087] During implementation, the three features calculated above, local energy, local quality, and spatial consistency, are normalized and weightedly combined into block feature values, which are used to determine the weight of each block during fusion (such as selecting the better part from the low-exposure or high-exposure image), and then perform weighted fusion.

[0088] In one embodiment, the block features of each block of the low-exposure original image are used to perform weight distribution and weighted fusion to obtain a low-exposure initial weight map, and the block features of each block of the high-exposure original image are used to perform weight distribution and weighted fusion to obtain a high-exposure initial weight map. This may include: based on the multi-feature weighted fusion principle and the calculated block features, quantizing the weights of each block of the low-exposure original image and the high-exposure original image, respectively, integrating the block features through an exponential function and the quantized weights, and obtaining the low-exposure initial weight map and the high-exposure initial weight map, respectively.

[0089] Specifically, based on the principle of multi-feature weighted fusion, the system quantifies the importance of each patch in the input image by integrating three key features: local energy, local quality, and spatial consistency. This system then constructs an adaptive weight distribution model, enabling the fusion process to intelligently distinguish the importance of different image regions. This model uses an exponential function to amplify the contribution of high-quality regions and suppress the influence of low-quality areas. This approach is adaptable to complex inspection scenarios, such as varying rail surface reflectance and ambient light interference.

[0090] In one embodiment, weight allocation and weighted fusion are performed using block features of each block of the low-exposure original image to obtain a low-exposure initial weight map, and weight allocation and weighted fusion are performed using block features of each block of the high-exposure original image to obtain a high-exposure initial weight map, which may include:

[0091] The low exposure initial weight map and the high exposure initial weight map are obtained according to the following formula:

[0092] (4)

[0093] Where W i is the low exposure initial weight map or the high exposure initial weight map, E i , Q i , SC i They represent the local energy value, local quality value, and spatial consistency value of the i-th block respectively.

[0094] In step 204 , based on the guided filtering principle, the low-exposure original image is used as a guide to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map, respectively, to obtain a low-exposure weight correction map and a high-exposure weight correction map.

[0095] Guided filtering is an effective image filtering technique, particularly suitable for tasks such as image denoising and image enhancement. Guided filtering is used to fuse specific features or details of the guide image into the input image while preserving edge information.

[0096] In one embodiment, based on the guided filtering principle, the low-exposure original image is used as a guide, and the low-exposure initial weight map and the high-exposure initial weight map are weight-corrected respectively to obtain a low-exposure weight correction map and a high-exposure weight correction map. This may include: using a preset filter window, a local linear transformation coefficient and the low-exposure original image to construct an objective function, and solving the objective function to obtain a weight correction map; the input of the objective function is the initial weight map, and the optimization goal of the objective function is to obtain a weight correction map by linearly fitting the input initial weight map within each filter window, so that the sum of the square errors between the weight correction map and the initial weight map is minimized.

[0097] The objective function can be expressed as follows:

[0098] (5)

[0099] Where i is the image pixel index;

[0100] ω k is the filter window centered at pixel k;

[0101] W i is the input low exposure initial weight map or high exposure initial weight map;

[0102] is the corrected weight map, i.e. the guide image I low The result of the local linear transformation of ;

[0103] a k 、b k is the local linear transformation coefficient, each filter window ω k Corresponding to a group a k 、b k ;

[0104] I low is the low-exposure original image, i.e. the guide image;

[0105] cov( ) ωk is the filter window ω k The covariance, cov(I low , W i ) ωk The guide image I low With the input initial weight map W i In the filter window ω k The covariance of is used to measure the linear correlation between the two in the local area;

[0106] Var( ) ωk is the filter window ω k The variance of Var(I low ) ωk The guide image I low In the filter window ω k The variance of is used to measure the degree of local grayscale change;

[0107] mean( ) ωk is the filter window ω k The mean value, mean(I low ) ωk The guide image I low In the filter window ω k The mean value within, mean(W i ) ωk The initial weight map W of the input i In the filter window ω k The mean within

[0108] ε is a set value, such as a very small positive number. ε is used to avoid the denominator being zero to ensure numerical stability.

[0109] In the final step 205 , the low-exposure weighted correction map and the high-exposure weighted correction map are weightedly fused pixel by pixel to obtain a fused image.

[0110] In one embodiment, performing pixel-by-pixel weighted fusion using the low-exposure weight correction map and the high-exposure weight correction map to obtain a fused image may include:

[0111] According to the following formula, the low-exposure weight correction map and the high-exposure weight correction map are weightedly fused to obtain the fused image:

[0112] (6)

[0113] Where, I F is the fused image, N=2, Indicates a low exposure weight correction map or a high exposure weight correction map, that is, a low exposure weight correction map or a high exposure weight correction map, I k Indicates the low-exposure original image or the high-exposure original image. For example, when k is 1 Represents the low exposure weight correction map, I k Indicates the low-exposure original image, for example, when k is 2 Represents the high exposure weight correction map, I k Represents the high-exposure original image, and the grayscale value of the image is used for calculation.

[0114] During implementation, the corrected weight map is weightedly fused with each pixel position of the input image to obtain the final fused image.

[0115] Next, select Q y The index is used to evaluate the quality of the fused image. It is a fusion quality measurement method based on SSIM, which indicates that the fused image I F The amount of structural information retained from the two source images. Q y is defined as follows:

[0116] (7)

[0117] In formula (7), A and B are source images (such as low-exposure original image and high-exposure original image);

[0118] F is the final fused image;

[0119] SSIM (A, F|ω) is the structural similarity index of images A and F within the local window ω. The higher the value, the more similar the structure.

[0120] SSIM(B, F|ω) is the structural similarity index of images B and F within the local window ω. The higher the value, the more similar the structure.

[0121] SSIM(A, B|ω) is the structural similarity index of images A and B within the local window ω. The higher the value, the more similar the structure.

[0122] λ(ω) is the weight coefficient, which is used to balance the contribution of A and B to the structural information of the fused image;

[0123] ω is the local window;

[0124] s is a local measure of image saliency;

[0125] λ(ω) is defined as follows:

[0126] (8)

[0127] Where S(A|ω) and S(B|ω) represent the saliency of source images A and B within the local window ω, respectively;

[0128] SSIM A,F is the structural similarity index of images A and F.

[0129] μ A 、μ F is the grayscale mean of image A in the local window ω, and the grayscale mean of image F in the local window ω;

[0130] δ AF is the covariance of image A and image F in the local window ω;

[0131] δ A is the variance of image A in the local window ω;

[0132] δ F is the variance of image F in the local window ω;

[0133] C2, C3, and C4 are predetermined constants.

[0134] Figure 6 Schematic diagram of a low-exposure original image and a high-exposure original image in an embodiment of the present invention, Figure 7 This is a schematic diagram comparing the fusion results of different algorithms in the embodiment of the present invention, refer to Figure 6 、 Figure 7 , compared and analyzed with the existing fusion algorithm, the fusion method proposed in the embodiment of the present invention can filter the spot noise, improve the imaging quality of the rail gauge angle, and achieve a better fusion effect.

[0135] Through comparative analysis, the fusion quality index Q of the fusion method proposed in the embodiment of the present invention isy The image fusion method based on dense scale-invariant features is DSIFT (Dense SIFT multi-exposure fusion), the multi-exposure image fusion method based on adaptive weights is MEFAW (Multi-Exposure_Image Fusion Based on the Adaptive Weights), the ghost-free multi-exposure image fusion method based on dense scale-invariant features is MEFDSIFT (Ghost-free Multi Exposure Image fusion technique using dense sift), the patch-wise approach (PWA) based multi-exposure image fusion method, the detail-enhanced multi-scale exposure fusion method based on YUV color space is DEM (Detail-enhanced Multi-scale Exposure Fusion in YUV Color Space), the fast multi-scale structural patch decomposition for multi-exposure image fusion method based on FMMEF (Fast multi-scale structural patch decomposition for multi-exposure image fusion), the multi-exposure image fusion method based on gradient domain is GD (Multi-exposure image fusion in gradient domain), and the image fusion method based on exposure ratio is MTI (Multi-Scale Fusion of Two Large-Exposure-Ratio The index values ​​of the fused images are 0.9, 0.76, 0.94, 0.81, 0.85, 0.87, 0.62, and 0.85, respectively. The fused images produced by the method of the embodiment of the present invention have high quality and the best effect, and retain the effective information of the rail images.

[0136] The present invention also provides a binocular multiple-exposure rail visual perception device, as described in the following embodiments. Because the principles of this device are similar to those of the binocular multiple-exposure rail visual perception method, the implementation of this device can be referenced to the implementation of the binocular multiple-exposure rail visual perception method, and any repetitions will not be repeated.

[0137] Figure 8 FIG. 1 is a schematic diagram of a binocular multiple exposure rail visual perception device according to an embodiment of the present invention. Figure 8 As shown, the apparatus 800 includes:

[0138] Image data acquisition module 801, used to obtain low-exposure original images and high-exposure original images of the same position of the rail taken by the binocular camera and the laser;

[0139] The block feature calculation module 802 is configured to divide the low-exposure original image and the high-exposure original image into multiple blocks and calculate the block features of each block. The block features include local energy, local quality, and spatial consistency. The local energy reflects the proportion of effective signals related to the laser stripe, the local quality reflects the similarity between multiple sub-regions based on the block division, and the spatial consistency reflects the vertical distance between the block and the energy center of the light stripe.

[0140] The initial weight map calculation module 803 is used to perform weight assignment and weighted fusion using the block features of each block of the low-exposure original image to obtain a low-exposure initial weight map, and to perform weight assignment and weighted fusion using the block features of each block of the high-exposure original image to obtain a high-exposure initial weight map;

[0141] The weight correction module 804 is used to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map based on the guided filtering principle, using the low-exposure original image as a guide, to obtain a low-exposure weight correction map and a high-exposure weight correction map;

[0142] The fusion processing module 805 is used to perform weighted fusion pixel by pixel using the low exposure weight correction map and the high exposure weight correction map to obtain a fused image.

[0143] In one embodiment, the image data acquisition module 801 is specifically configured to:

[0144] Acquire multiple exposure images of the same rail position taken by the binocular camera and laser. The multiple exposure images include a low-exposure image from the left camera, a high-exposure image from the left camera, a low-exposure image from the right camera, and a high-exposure image from the right camera.

[0145] Using the preset quality assessment method, images with higher quality scores are screened out from the low-exposure images of the left camera and the low-exposure images of the right camera and recorded as low-exposure original images, and images with higher quality scores are screened out from the high-exposure images of the left camera and the high-exposure images of the right camera and recorded as high-exposure original images.

[0146] In one embodiment, the preset quality assessment method includes: calculating an image entropy value, and treating an image with a larger image entropy value as an image with a higher quality score.

[0147] In one embodiment, the block feature calculation module 802 is specifically configured to:

[0148] The local energy of each block is calculated as follows:

[0149] The first threshold is used to distinguish valid signals from invalid areas, and the valid signal ratio of each block is calculated.

[0150] In one embodiment, the block feature calculation module 802 is specifically configured to:

[0151] According to the following formula, the first threshold is used to distinguish valid signals from invalid areas, and the valid signal ratio of each block is calculated:

[0152] ;

[0153] Where, E i represents the local energy value of the i-th block, α and β are coefficients, μ i is the grayscale mean of all pixels in the i-th block, p i 、ratio i , γ i (x, y) is the intermediate calculation amount, I i (x, y) represents the grayscale of the pixel (x, y) in the i-th block.

[0154] In one embodiment, the block feature calculation module 802 is specifically configured to:

[0155] The local quality of each block is calculated as follows:

[0156] Each block is divided into multiple sub-regions, and the structural similarity SSIM value between the sub-regions is calculated.

[0157] In one embodiment, the block feature calculation module 802 is specifically configured to:

[0158] Each block is divided into multiple sub-regions according to the following formula, and the SSIM values ​​between sub-regions are calculated:

[0159] ;

[0160] Where Q i represents the local quality value of the i-th block, Slide represents the number of sub-regions divided by the block, S(u,v) represents the structural similarity measure between the u-th sub-region and the v-th sub-region, represents the variance of the u-th sub-region, represents the variance of the vth sub-region, δ uv represents the covariance between the u-th sub-region and the v-th sub-region, and C1 is a preset constant.

[0161] In one embodiment, the block feature calculation module 802 is specifically configured to:

[0162] The spatial consistency of each block is calculated as follows:

[0163] ;

[0164] Where, SC i represents the spatial consistency value of the i-th block, is the vertical position of the ith block, and H is the vertical position of the light band energy center.

[0165] In one embodiment, the initial weight map calculation module 803 is specifically configured to:

[0166] Based on the principle of multi-feature weighted fusion and the calculated block features, the weights of each block in the low-exposure original image and the high-exposure original image are quantized respectively. The block features are integrated through the exponential function and the quantized weights to obtain the low-exposure initial weight map and the high-exposure initial weight map, respectively.

[0167] In one embodiment, the initial weight map calculation module 803 is specifically configured to:

[0168] The low exposure initial weight map and the high exposure initial weight map are obtained according to the following formula:

[0169] ;

[0170] Where W i is the low exposure initial weight map or the high exposure initial weight map, E i , Q i , SC i They represent the local energy value, local quality value, and spatial consistency value of the i-th block respectively.

[0171] In one embodiment, the weight modification module 804 is specifically configured to:

[0172] Using the preset filter window, local linear transformation coefficients and low-exposure original image, an objective function is constructed, and the objective function is solved to obtain the weight correction map; the input of the objective function is the initial weight map, and the optimization goal of the objective function is to obtain the weight correction map by linearly fitting the input initial weight map within each filter window, so as to minimize the sum of the square errors between the weight correction map and the initial weight map.

[0173] In one embodiment, the objective function is expressed as follows:

[0174] ;

[0175] Where i is the image pixel index, ω k is the filter window centered at pixel k, W i is the input low exposure initial weight map or high exposure initial weight map, is the modified weight map, a k 、b k is the local linear transformation coefficient, I low is the low-exposure original image, cov( ) ωk is the filter window ω k The covariance of Var( ) ωkis the filter window ω k The variance, mean( ) ωk is the filter window ω k , ε is a set value, and ε is used to avoid the denominator being zero.

[0176] In one embodiment, the fusion processing module 805 is specifically configured to:

[0177] According to the following formula, the low-exposure weight correction map and the high-exposure weight correction map are weightedly fused to obtain the fused image:

[0178] ;

[0179] Where, I F is the fused image, N=2, Indicates a low exposure weight correction map or a high exposure weight correction map, I k Indicates low-exposure original image or high-exposure original image.

[0180] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned binocular multiple-exposure rail visual perception method is implemented.

[0181] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the above-mentioned binocular multiple-exposure rail visual perception method.

[0182] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the above-mentioned binocular multiple exposure rail visual perception method.

[0183] The method according to the embodiment of the present invention has the following advantages:

[0184] (1) In the embodiment of the present invention, a high-quality rail visual perception method based on binocular multiple exposure is proposed, which can effectively solve the problem of rail image quality degradation caused by ambient light interference, insufficient dynamic range and differences in surface reflection characteristics.

[0185] (2) In an embodiment of the present invention, it is proposed to fuse high-quality rail images of multiple exposures, and improve the fusion quality of rail images based on the local energy, local quality and spatial consistency feature distribution of each block in the image.

[0186] (3) In the embodiments of the present invention, a binocular camera is combined with multiple exposure technology. This method is not only applicable to rail surface defect detection, but can also be expanded to other fields such as switch rail status monitoring and track foreign object intrusion detection. It can improve detection stability in complex environments, increase detection efficiency, reduce operation and maintenance costs, and promote the intelligent upgrade of rail transportation.

[0187] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0188] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0191] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A binocular multiple exposure rail visual perception method, characterized in that: include: Obtain low-exposure and high-exposure original images of the same rail position taken by the binocular camera and laser; The low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated. The block features include local energy, local quality, and spatial consistency. Local energy reflects the proportion of effective signals related to the laser stripe, local quality reflects the similarity between multiple sub-regions based on the block division, and spatial consistency reflects the vertical distance between the block and the energy center of the light stripe. The low-exposure initial weight map is obtained by weighting and fusion using the block features of each block of the low-exposure original image. The high-exposure initial weight map is obtained by weighting and fusion using the block features of each block of the high-exposure original image. Based on the guided filtering principle, the low-exposure original image is used as a guide to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map, respectively, to obtain the low-exposure weight correction map and the high-exposure weight correction map; The low-exposure weight correction map and the high-exposure weight correction map are used to perform weighted fusion pixel by pixel to obtain the fused image.

2. The method according to claim 1, wherein The low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated, including: The local energy of each block is calculated as follows: ； Where, E i Indicates the i The local energy value of each block, α and β is the coefficient, p i represents the effective energy, ratio i Characterizes the effective signal pixel ratio, μ i For the i The grayscale mean of all pixels in a block, the first threshold is set based on the grayscale mean of the pixels in each block, wherein the first threshold is used to distinguish between valid signals and invalid areas, and the effective signal ratio of each block is calculated. I i ( x , y ) indicates the i Pixels in the block ( x , y ) grayscale, Equal to 1 means I i ( x , y ) is a valid signal, Equal to 0 means I i ( x , y ) is an invalid area, N is the number of pixels in the block.

3. The method according to claim 1, wherein The low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated, including: The local quality of each block is calculated as follows: ； Where, Q i Indicates the i The local quality value of each block; each block is divided into multiple sub-regions, and the SSIM value between sub-regions is calculated S ( u , v ), S ( u , v ) indicates the u sub-regions and v The SSIM value of the sub-region, Slide Indicates the number of sub-regions divided by the block, Indicates the u The variance of the sub-region, Indicates the v The variance of the sub-region, δ uv Indicates the u sub-regions and v The covariance of the subregions, C 1 is the preset constant.

4. The method according to claim 1, wherein The low-exposure original image and the high-exposure original image are divided into multiple blocks, and the block features of each block are calculated, including: The spatial consistency of each block is calculated as follows: ； Where, SC i Indicates the i The spatial consistency value of each block, For the i The vertical position of the block, H It is the vertical position of the energy center of the light band.

5. The method according to claim 1, wherein The low-exposure initial weight map is obtained by weighting and fusion of the block features of each block of the low-exposure original image. The high-exposure initial weight map is obtained by weighting and fusion of the block features of each block of the high-exposure original image, including: Based on the multi-feature weighted fusion principle and the calculated block features, the weights of each block of the low-exposure original image and the high-exposure original image are quantized respectively. The block features are integrated through the exponential function and the quantized weights. The low-exposure initial weight map and the high-exposure initial weight map are obtained according to the following formulas: ； Where, W i is the low exposure initial weight map or the high exposure initial weight map, E i 、 Q i 、 SC i Respectively represent i The local energy value, local quality value, and spatial consistency value of each block.

6. The method according to claim 1, wherein Based on the guided filtering principle, the low-exposure original image is used as a guide to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map, respectively, to obtain the low-exposure weight correction map and the high-exposure weight correction map, including: Using the preset filter window, local linear transformation coefficients, and the low-exposure original image, an objective function is constructed and solved to obtain a weight correction map. The input of the objective function is the initial weight map. The optimization goal of the objective function is to obtain a weight correction map by linearly fitting the input initial weight map within each filter window, so that the sum of the squared errors between the weight correction map and the initial weight map is minimized. The objective function is expressed as follows: ； Where, i is the image pixel index, ω k Pixel k The filter window centered at W i is the input low exposure initial weight map or high exposure initial weight map, is the modified weight map, a k 、 b k is the local linear transformation coefficient, I low This is the original low-exposure image. cov ( ) ωk is the filter window ω k The covariance of Var ( ) ωk is the filter window ω k The variance of mean ( ) ωk is the filter window ω k The mean of ε To set the value, ε Used to avoid denominators being zero; The fused image is obtained by performing pixel-by-pixel weighted fusion using the low-exposure weighted correction map and the high-exposure weighted correction map, including: According to the following formula, the low-exposure weight correction map and the high-exposure weight correction map are weightedly fused to obtain the fused image: ； Where, I F To fuse the images, N =2, Indicates a low exposure weight correction map or a high exposure weight correction map, I k Indicates low-exposure original image or high-exposure original image.

7. A binocular multiple exposure rail visual perception device, characterized in that: include: Image data acquisition module, used to obtain low-exposure original images and high-exposure original images of the same position on the rail taken by the binocular camera and laser; A block feature calculation module is used to divide the low-exposure original image and the high-exposure original image into multiple blocks and calculate the block features of each block; the block features include local energy, local quality, and spatial consistency; local energy reflects the proportion of effective signals related to the laser stripe, local quality reflects the similarity between multiple sub-regions based on the block division, and spatial consistency reflects the vertical distance between the block and the energy center of the light stripe; The initial weight map calculation module is used to use the block features of each block of the low-exposure original image to perform weight distribution and weighted fusion to obtain a low-exposure initial weight map, and to use the block features of each block of the high-exposure original image to perform weight distribution and weighted fusion to obtain a high-exposure initial weight map; The weight correction module is used to perform weight correction on the low-exposure initial weight map and the high-exposure initial weight map based on the guided filtering principle, and obtain the low-exposure weight correction map and the high-exposure weight correction map respectively; The fusion processing module is used to perform weighted fusion pixel by pixel using the low-exposure weight correction map and the high-exposure weight correction map to obtain a fused image.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images