Image stitching method and device based on linear array camera, equipment and storage medium

Abstract

This invention discloses an image stitching method, apparatus, device, and storage medium based on a line scan camera. The method includes: obtaining the starting and ending geographic coordinates corresponding to an initial image sequence, and determining the geographic distance corresponding to the initial image sequence; stitching together first-type images in the initial image sequence to obtain a second-type image corresponding to the initial image sequence; obtaining the image length of the second-type image, and adjusting the second-type image according to the geographic distance and the image length to obtain a target image corresponding to the initial image sequence. This invention determines the geographic distance corresponding to the initial image sequence based on the starting and ending geographic coordinates in the initial image sequence, and adjusts the second-type image obtained by stitching together each first-type image in the initial image sequence according to the geographic distance, so that images of the same resolution represent the same length information, thereby enabling the stitching of images of the same resolution.

Description

Technical Field

[0001] This invention relates to the field of image acquisition technology, and specifically to an image stitching method, apparatus, device, and storage medium based on a line scan camera. Background Technology

[0002] With the widespread adoption of machine vision, line scan cameras are increasingly recognized by vision engineers and end users. Due to their high spatial resolution, line scan cameras enable high-precision measurements, making them widely used for non-contact one-dimensional measurement. Driven by the need for cost control in scanning imaging, the requirement for clear imaging of complex target surfaces, and the need for improved positioning accuracy of fine mobile platforms, line scan cameras are increasingly used for scanning and imaging designated target surfaces.

[0003] In pavement inspection applications, a line scan camera is installed in the pavement inspection device. As the pavement inspection device moves, it scans the road surface to obtain multiple bar images. These multiple bar images are then stitched together to reach the required number or length of stitched images before outputting a complete image.

[0004] Existing line scan cameras rely on encoder pulse counts for triggering, recording the movement distance of the pavement detection device through encoder pulses. However, due to the influence of the road environment and the movement speed of the pavement detection device, the device cannot move at a constant speed. This causes the encoder pulse count of the line scan camera to be inconsistent with the actual movement distance of the pavement detection device. In other words, images with the same resolution record different pavement lengths, resulting in subsequent images of the same resolution not being able to be stitched together correctly. Summary of the Invention

[0005] This invention provides an image stitching method, apparatus, device, and storage medium for a line scan camera to solve the problem that the image length and the recorded geographical length are different in existing pavement inspection devices because the pavement inspection device cannot move at a uniform speed.

[0006] On one hand, embodiments of the present invention provide an image stitching method based on a linear scan camera, applied to a pavement inspection robot, the method comprising:

[0007] Obtain the starting and ending geographic coordinates corresponding to the initial image sequence, determine the geographic distance corresponding to the initial image sequence, wherein the initial image sequence includes multiple first-type images, and wherein the first-type images are strip images;

[0008] By stitching together the first type of images in the initial image sequence, a second type of image corresponding to the initial image sequence is obtained;

[0009] The image length of the second type of image is obtained, and the second type of image is adjusted according to the geographical distance and the image length to obtain the target image corresponding to the initial image sequence.

[0010] In some embodiments of the present invention,

[0011] The step of obtaining the image length of the second type of image, and adjusting the second type of image according to the geographical distance and the image length to obtain the initial image sequence includes:

[0012] Obtain the image length of the second type of image;

[0013] Obtain the standard image length corresponding to the geographical distance;

[0014] The adjustment ratio is determined based on the difference between the image length and the standard image length;

[0015] The second type of image is stretched or compressed according to the adjustment ratio, and the stretched or compressed second type of image is used as the target image corresponding to the initial image sequence.

[0016] In some embodiments of the present invention, the step of stitching together each of the first type of images in the initial image sequence to obtain the second type of image corresponding to the initial image sequence includes:

[0017] The target heading angle is determined based on the statistical characteristics of the heading angles of each of the first type of images. The statistical characteristics include at least one of the median, mode, mean, and expected value of each heading angle.

[0018] Calculate the difference between the target heading angle and each heading angle to obtain the rotation angle corresponding to each of the first type of images;

[0019] The first type of images are stitched together according to the rotation angle corresponding to each first type of image to obtain the second type of image.

[0020] In some embodiments of the present invention, obtaining the starting and ending geographic coordinates corresponding to the initial image sequence and determining the geographic distance corresponding to the initial image sequence includes:

[0021] Obtain the first and last acquisition times corresponding to the first type of image in the initial image sequence;

[0022] Based on a preset address list, obtain the starting geographic coordinates corresponding to the first collection time and the ending geographic coordinates corresponding to the last collection time.

[0023] Calculate the distance between the starting geographic coordinates and the ending geographic coordinates to obtain the geographic distance corresponding to the initial image sequence.

[0024] In some embodiments of the present invention, obtaining the first acquisition time and the last acquisition time corresponding to the first type of image in the initial image sequence includes:

[0025] Obtain the global encoding range corresponding to the initial image sequence, and obtain the start encoding and end encoding corresponding to the global encoding range;

[0026] Obtain the first acquisition time corresponding to the start code in the preset image list, and obtain the last acquisition time corresponding to the end code in the preset image list.

[0027] In some embodiments of the present invention, before the steps of obtaining the starting and ending geographic coordinates corresponding to the initial image sequence and determining the geographic distance corresponding to the initial image sequence, the method includes:

[0028] The movement speed of the pavement inspection robot is obtained, a first type of image is acquired based on the movement speed, a code corresponding to the first type of image is assigned, and the acquisition time of the first type of image and the code corresponding to the acquisition time are stored in a preset image list.

[0029] If the number of images of the first type in the preset image list reaches a preset threshold, then each image of the first type is obtained, and the target code corresponding to each image of the first type is obtained;

[0030] Each of the first type of images is associated with the corresponding target code to obtain an initial image sequence.

[0031] In some embodiments of the present invention, before the steps of obtaining the starting and ending geographic coordinates corresponding to the initial image sequence and determining the geographic distance corresponding to the initial image sequence, the method includes:

[0032] Obtain the global encoding range corresponding to the initial image sequence;

[0033] The global coding range is divided into multiple local coding ranges according to a preset coding interval;

[0034] Obtain the local geographic distance corresponding to each of the local coding ranges, stitch together each of the first type images within each of the local coding ranges to obtain a local second type image, and obtain the local image length corresponding to each of the local second type images;

[0035] Based on the local geographic distance and the local image length, each of the local second-type images is adjusted to obtain the adjusted image corresponding to each of the local second-type images;

[0036] The target image is obtained by stitching together the adjusted images.

[0037] On the other hand, embodiments of the present invention provide an image stitching device based on a line scan camera, the device comprising:

[0038] The address module is used to obtain the starting and ending geographic coordinates corresponding to the initial image sequence and determine the geographic distance corresponding to the initial image sequence. The initial image sequence includes multiple first-type images, wherein the first-type images are strip images.

[0039] The stitching module is used to stitch together each of the first type of images in the initial image sequence to obtain the second type of image corresponding to the initial image sequence;

[0040] An adjustment module is used to obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and the image length to obtain the target image corresponding to the initial image sequence.

[0041] On the other hand, embodiments of the present invention provide an image stitching device based on a line scan camera, including a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to perform the operations in the image stitching method based on the line scan camera.

[0042] On the other hand, embodiments of the present invention provide a storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps in the image stitching method based on a line scan camera.

[0043] This invention obtains the starting and ending geographic coordinates of an initial image sequence, determines the geographic distance of the initial image sequence, and the initial image sequence includes multiple first-type images, wherein the first-type images are strip images; the first-type images in the initial image sequence are stitched together to obtain a second-type image corresponding to the initial image sequence; the image length of the second-type image is obtained, and the second-type image is adjusted according to the geographic distance and the image length to obtain the target image corresponding to the initial image sequence; this invention determines the geographic distance of the initial image sequence based on the starting and ending geographic coordinates in the initial image sequence, and adjusts the second-type image obtained by stitching together each first-type image in the initial image sequence according to the geographic distance, so that images of the same resolution represent the same length information, thereby enabling the stitching of images of the same resolution. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention;

[0046] Figure 2 This is an illustration of an initial image sequence provided in an embodiment of the present invention;

[0047] Figure 3 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention.

[0048] Figure 4 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention for adjusting the initial image;

[0049] Figure 5 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention, which involves segmenting and adjusting the image.

[0050] Figure 6 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention, illustrating the initial image acquisition process.

[0051] Figure 7 This is a flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention for determining geographical distance;

[0052] Figure 8 This is a schematic diagram of an embodiment of the image stitching device based on a line scan camera provided in this invention.

[0053] Figure 9 This is a schematic diagram of an embodiment of the image stitching device based on a line scan camera provided in this invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention provides an image stitching method, apparatus, device, and storage medium based on a line scan camera. According to the embodiments of the image stitching method based on a line scan camera provided by this invention, it should be noted that the steps shown in the flowcharts can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the described or shown steps can be executed in a different order than that shown here.

[0056] In some embodiments of the present invention, the image stitching method based on a line scan camera can be applied to at least one of a computer device and a terminal device. The computer device can be an independent server, a server network, or a server cluster, such as a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server consists of a large number of computers or network servers based on cloud computing. The terminal device includes, but is not limited to, smartphones, tablets, and PCs. In some embodiments of the present invention, when the image stitching method based on a line scan camera is applied to a computer device and a terminal device, communication between the terminal device and the computer device can be achieved through any communication method. This communication method includes, but is not limited to, mobile communication based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP / IP Protocol Suite (TCP / IP) and User Datagram Protocol (UDP). In some embodiments of the present invention, the image stitching method based on a line scan camera can also be applied to inspection equipment, such as pavement inspection robots and tunnel inspection robots. This embodiment of the present invention will illustrate the application of the image stitching method based on a line scan camera to a pavement inspection robot as an example. Figure 1 As shown, Figure 1 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided by the present invention. The image stitching method based on a line scan camera shown includes steps 101 to 103:

[0057] Step 101: Obtain the starting and ending geographic coordinates corresponding to the initial image sequence, and determine the geographic distance corresponding to the initial image sequence.

[0058] The initial image sequence includes multiple first-type images. In some embodiments of the present invention, the first-type images refer to one-dimensional images, i.e., strip images. For example... Figure 2 As shown, where Figure 2 Figure (a) is a schematic diagram of an initial image sequence provided in an embodiment of the present invention. The initial image sequence shown is a set of ordered first-class images arranged in the order of acquisition time, where N is the number of first-class images in the initial image sequence. For example, N can be 500. In some embodiments of the present invention, the number N of images in the initial image sequence can be set according to the actual application scenario of the image stitching method based on a line scan camera.

[0059] In some embodiments of the present invention, the initial image sequence may be a collection of all images acquired by the pavement inspection robot during a single pavement inspection, such as... Figure 2 As shown in Figure (a), this can be understood as compressing or stretching all acquired images; in some embodiments of the present invention, the initial image sequence may be a subset of all images acquired by the pavement inspection robot during a single pavement inspection, such as... Figure 2 As shown in Figure (b), all the acquired images are divided into two subsets to obtain two initial image sequences, which can be understood as segmenting and compressing or stretching all the acquired images.

[0060] The starting geographic coordinates refer to the geographic coordinates corresponding to the first image of type 1 in the initial image sequence, and the ending geographic coordinates refer to the geographic coordinates corresponding to the last image of type 1 in the initial image sequence. In some embodiments of the present invention, the geographic coordinates can be acquired based on the positioning sensors in the pavement inspection robot.

[0061] Geographic distance is the true distance represented by the initial image sequence, and true distance refers to the actual distance traveled by the pavement inspection robot.

[0062] In some embodiments of the present invention, when the pavement inspection robot moves at a constant speed, the true distance represented by the initial image sequence is consistent with the true length represented by the total length of the initial image sequence. However, due to the influence of the road environment and the moving speed of the pavement inspection device, the true length of the initial image sequence acquired by the line scan camera is inconsistent with the geographical distance corresponding to the initial image sequence. Therefore, based on the difference between the true length and the geographical distance, each initial image in the initial image sequence can be stretched or compressed so that the true length of the initial image sequence is the same as the geographical distance.

[0063] Step 102: Concatenate the first type of images in the initial image sequence to obtain the second type of images corresponding to the initial image sequence.

[0064] The second type of image refers to any one of the following: a two-dimensional planar image, a three-dimensional three-dimensional image, and a depth image obtained by stitching together multiple first-type images. In some embodiments of the present invention, all first-type images in the initial image sequence can be stitched together to obtain the second-type image.

[0065] In some embodiments of the present invention, when stitching the first type of images, the target heading angle of the first type of images is obtained as the rotation direction of the first type of images, and the first type of images are stitched together. Here, the heading angle refers to the angle between the projection of the longitudinal axis of the pavement inspection robot onto the horizontal plane and the geographic meridian. In some embodiments of the present invention, the heading angle ranges from -180° to 180°. The specific stitching method is as follows: Figure 3 As shown, Figure 3 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention. The method for stitching images shown includes steps 301 to 303:

[0066] Step 301: Determine the target heading angle based on the statistical characteristics of the heading angles of each first-class image.

[0067] Statistical characteristics include at least one of the following for each heading angle: median, mode, mean, and expected value.

[0068] In some embodiments of the present invention, the heading angle corresponding to the first type of image at the center position in the initial image sequence can also be used as the heading angle, wherein the center position refers to the sequence number or acquisition time corresponding to the first type of image being the median of the sequence number or all acquisition times in the initial image sequence.

[0069] Step 302: Calculate the difference between the target heading angle and each heading angle to obtain the rotation angle corresponding to each first type of image.

[0070] The rotation angle can be either clockwise or counterclockwise.

[0071] In some embodiments of the present invention, the difference between the target heading angle and each heading angle can be used as the offset corresponding to each adjusted image.

[0072] Step 303: The first type of images are stitched together according to the rotation angle corresponding to each first type of image to obtain the second type of image.

[0073] In some embodiments of the present invention, step 303 includes: rotating each first-type image according to the rotation angle corresponding to each first-type image; performing image enhancement, image denoising, and other image processing on the rotated first-type images; and stitching the processed first-type images together to obtain a second-type image. In some embodiments of the present invention, image enhancement may include contrast enhancement, brightness enhancement, sharpness enhancement, grayscale equalization, etc., and image denoising may include image noise reduction, image dehazing, etc.

[0074] Step 103: Obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and image length to obtain the target image corresponding to the initial image sequence.

[0075] The target image is the second type of image after adjusting its length.

[0076] In some embodiments of the present invention, adjusting includes compressing the length of an image or stretching the length of an image.

[0077] In some embodiments of the present invention, the target image may be a second type of image after the length of the compressed image or the length of the stretched image.

[0078] In this embodiment of the invention, the geographic distance corresponding to the initial image sequence is determined based on the starting and ending geographic coordinates in the initial image sequence. The second type of image obtained by stitching together the first type of images in the initial image sequence is adjusted so that images of the same resolution represent the same length information. Thus, second type of images of the same resolution can be stitched together, and the true width of pavement defects can be accurately determined based on the output image.

[0079] In some embodiments of the present invention, the standard length of the image can be determined based on geographical distance, the difference between the target length and the image length of the second type of image can be calculated, and the second type of image obtained by stitching together each first type of image in the initial image sequence can be stretched or compressed based on the difference, specifically as follows: Figure 4 As shown, Figure 4 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided by the present invention for adjusting a second type of image. The method for adjusting the second type of image shown includes steps 401 to 404:

[0080] Step 401: Obtain the image length of the second type of image.

[0081] In some embodiments of the present invention, the image width value or image length value of the second type of image can be obtained as the image length of the initial image sequence.

[0082] Step 402: Obtain the standard image length corresponding to the geographical distance.

[0083] In some embodiments of the present invention, preset mapping parameters can be obtained, and the standard image length corresponding to the geographical distance can be calculated based on the geographical distance and the preset mapping parameters. For example, the standard image length corresponding to the geographical distance can be obtained by calculating the product between the geographical distance and the preset mapping parameters.

[0084] In some embodiments of the present invention, the setting parameters of the line scan camera can also be obtained, and the standard image length can be obtained based on the setting parameters and the geographical distance. For example, the standard image length can be obtained by calculating the product of the geographical distance and the imaging magnification in the line scan camera.

[0085] Step 403: Determine the adjustment ratio based on the difference between the image length and the standard image length.

[0086] The adjustment ratio can be either the compression ratio of the image length or the stretch ratio of the image length.

[0087] In some embodiments of the present invention, step 403 includes: obtaining the difference between the image length and the standard image length by subtracting the image length from the image length, and determining the adjustment ratio by calculating the difference / total image length * 100%.

[0088] Step 404: Stretch or compress the second type of image according to the adjustment ratio, and use the stretched or compressed second type of image as the target image corresponding to the initial image sequence.

[0089] In some embodiments of the present invention, when the adjustment ratio is less than 0, it indicates that the image length is less than the standard image length, and the second type of image is stretched to obtain the target image corresponding to the initial image sequence; if the adjustment ratio is greater than 0, the second type of image is compressed to obtain the target image corresponding to the initial image sequence.

[0090] In some embodiments of the present invention, the theoretical distance corresponding to the image length can also be obtained, the distance difference between the theoretical distance and the geographical distance can be calculated, the adjustment ratio corresponding to the distance difference can be obtained, and the second type of image can be stretched or compressed according to the adjustment ratio to obtain the target image corresponding to the initial image sequence. The method for obtaining the theoretical distance corresponding to the image length is similar to the method for calculating the true distance of the initial image sequence in step 102, and will not be described again here.

[0091] This invention provides a convenient method for determining the standard length of an image based on geographical distance, calculates the difference between the target length and the image length of the second type of image, and stretches or compresses the second type of image based on the difference. By adjusting the ratio to stretch or compress the second type of image, images with the same resolution can represent the same length information.

[0092] In some embodiments of the present invention, in order to improve the accuracy of the target image, the image length is adjusted in segments, specifically as follows: Figure 5 As shown, Figure 5 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention, showing the method for segmented image adjustment, which includes steps 501 to 505:

[0093] Step 501: Obtain the global encoding range corresponding to the initial image sequence.

[0094] The encoding is used to record the acquisition order of each initial image. In some embodiments of the present invention, the encoding can be a sequence number, for example, setting the encoding corresponding to the first acquired first-type image to 01. In some embodiments of the present invention, during the acquisition of the initial image sequence, for each acquired first-type image, the current largest encoding in the preset image list is obtained, and the current largest encoding + 1 is used as the encoding of the first-type image. This encoding is then associated with the acquisition time of the first-type image and written into the preset image list. The preset image list is used to store the acquisition time of each first-type image and the encoding of each first-type image during the image acquisition process by the pavement inspection robot.

[0095] In some embodiments of the present invention, the encoding range refers to the encoding range consisting of the minimum and maximum encodings in a preset image list.

[0096] In some embodiments of the present invention, the completion of the initial image sequence acquisition can be determined based on the number of acquisitions. If the initial image sequence acquisition is complete, the global encoding range corresponding to the initial image sequence is obtained. In some embodiments of the present invention, the number of acquisitions can be determined based on the minimum and maximum codes in a preset image list. For example, the difference between the minimum and maximum codes can be calculated, and the difference + 1 can be used as the number of acquisitions.

[0097] In some embodiments of the present invention, the number of acquisitions can also be read and recorded. Specifically, when the pavement inspection robot calls the line scan camera to take pavement pictures, the acquisition count is set to 1 each time the line scan camera starts to take the first first type of image. Thereafter, the acquisition count is incremented by 1 for each first type of image taken by the line scan camera, and the recorded acquisition count is compared with the preset acquisition count. If the recorded acquisition count reaches the preset acquisition count, the global encoding range corresponding to the initial image sequence is obtained, and the recorded acquisition count is set to 0. If the recorded acquisition count does not reach the preset acquisition count, image acquisition continues through the line scan camera, and the acquisition count continues to be recorded.

[0098] Step 502: Divide the global encoding range into multiple local encoding ranges according to the preset encoding interval.

[0099] The preset encoding interval is used to indicate the number of images corresponding to each segment in the segmented adjustment image. For example, when the preset encoding interval is 10 and the global encoding range is 1 to 500, the global encoding range is divided into 50 local encoding ranges according to the order of encoding from largest to smallest or smallest to largest, and the number of first-type images in each local encoding range is 10. When the preset encoding interval is 10, the geographic coordinates corresponding to the smallest and largest codes in every 10 codes in the global encoding range are obtained to obtain the local geographic distances corresponding to these 10 codes. These 10 first-type images are then stitched together to obtain local second-type images. The image length of each local second-type image is obtained, and the image length of each local second-type image is stretched or compressed according to the local geographic distance to obtain the adjusted image corresponding to each local second-type image. It should be noted that the preset encoding interval is not limited in the embodiments of the present invention and can be adjusted according to the actual application scenario of the image stitching method based on a line scan camera.

[0100] Step 503: Obtain the local geographic distance corresponding to each local coding range, stitch together each first-class image within each local coding range to obtain a local second-class image, and obtain the local image length corresponding to each local second-class image.

[0101] In some embodiments of the present invention, step 503 includes: obtaining the minimum and maximum codes in each local coding range, obtaining the acquisition time corresponding to the minimum code, and obtaining the geographic coordinates corresponding to the acquisition time; obtaining the acquisition time corresponding to the maximum code in each local coding range, and obtaining the geographic coordinates corresponding to the acquisition time; calculating the difference between the two geographic coordinates as the local geographic distance corresponding to each local coding range; obtaining the first type of image corresponding to each code in each local coding range; stitching each first type of image according to the image stitching method described in steps 301 to 303 to obtain a local second type of image; and obtaining the theoretical distance corresponding to the image length based on the image length of the local second type of image.

[0102] Step 504: Adjust each local second-type image according to the local geographical distance and local image length to obtain the adjusted image corresponding to each local second-type image.

[0103] In some embodiments of the present invention, the method for obtaining the locally adjusted image in step 503 is similar to the method for obtaining the target image described in steps 403 to 404, and will not be repeated here.

[0104] In some embodiments of the present invention, the local second-class images spliced ​​from each first-class image corresponding to the local coding range can be stretched or compressed according to the local geographical distance and the local image length to obtain the adjusted images corresponding to each local second-class image.

[0105] Step 505: Stitch together the adjusted images to obtain the target image.

[0106] In an embodiment of the present invention, after the linear array camera completes one shot, it divides multiple local coding ranges, stretches or compresses the local second-type images obtained by stitching together the first-type images corresponding to each code in each local coding range, and obtains the adjusted images corresponding to each local second-type image. By stitching together the adjusted images, the target image is obtained, thereby segmenting and adjusting the acquired initial image sequence to improve the accuracy of the target image.

[0107] In some embodiments of the present invention, during the initial image acquisition, each initial image is encoded and allocated according to the acquisition time of the initial image, such as... Figure 6 As shown, Figure 6 This is a schematic flowchart of an embodiment of the image stitching method based on a line scan camera provided in this invention, showing the initial image acquisition method including steps 601 to 603:

[0108] Step 601: Obtain the moving speed of the pavement inspection robot, collect a first type of image based on the moving speed, assign a code corresponding to the first type of image, and store the collection time of the first type of image and the code corresponding to the collection time into a preset image list.

[0109] In some embodiments of the present invention, the moving speed of the pavement inspection robot can be obtained from the sensors installed on the pavement inspection robot. The encoder installed on the pavement inspection robot generates pulses according to the moving speed. The line scan camera on the pavement inspection robot is triggered according to the pulses to acquire a first type of image. The code corresponding to the first type of image is assigned. The acquisition time of the first type of image and the code corresponding to the acquisition time are stored in a preset image list. The sequence number of the pulse corresponding to the triggering of the line scan camera on the pavement inspection robot is used as the code corresponding to the acquired first type of image.

[0110] Step 602: If the first type of image in the preset image list reaches the preset threshold, then obtain each first type of image and obtain the target code corresponding to each first type of image.

[0111] Step 603: Associate each first-class image with the corresponding target code to obtain an initial image sequence.

[0112] In some embodiments of the present invention, the number of first-type images in a preset image list can also be obtained. If the number of first-type images does not reach a preset threshold, the initial images are stitched together according to the stitching method shown in steps 301 to 303 to obtain a stitched image. The first-type image acquisition steps of steps 601 to 603 are continued to obtain new first-type images. The new first-type images are stitched together according to the stitching method shown in steps 301 to 303 to obtain a new stitched image. If an image acquisition end command is received, the stitched images are combined to obtain an initial image sequence. The initial image sequence, as well as the preset image list and preset address list corresponding to each stitched image, are transmitted to a computer device.

[0113] In some embodiments of the present invention, the geographical distance corresponding to the initial image sequence can be obtained based on the initial image sequence, a preset image list, and a preset address list, specifically as follows: Figure 7 As shown, Figure 7 This is a flowchart illustrating an embodiment of the image stitching method based on a line scan camera provided in this invention for determining geographical distance. The method for determining geographical distance includes steps 701 to 703:

[0114] Step 701: Obtain the first acquisition time and the last acquisition time corresponding to the first type of image in the initial image sequence.

[0115] The first acquisition time indicates the acquisition time of the first image of type 1 in the initial image sequence, and the last acquisition time indicates the acquisition time of the last image of type 1 in the initial image sequence.

[0116] In some embodiments of the present invention, step 701 includes: obtaining the first acquisition time corresponding to the start code and the last acquisition time corresponding to the stop code based on the start code and stop code corresponding to the initial image sequence. Specifically, it includes steps a1 to a2:

[0117] Step a1: Obtain the global encoding range corresponding to the initial image sequence, and obtain the start encoding and end encoding corresponding to the global encoding range.

[0118] The starting code refers to the minimum code in the global coding range, and the ending code refers to the maximum code in the global coding range.

[0119] Step a2: Obtain the first acquisition time corresponding to the start code in the preset image list, and obtain the last acquisition time corresponding to the end code in the preset image list.

[0120] Step 702: Based on the preset address list, obtain the starting geographic coordinates corresponding to the first collection time and the ending geographic coordinates corresponding to the last collection time.

[0121] Step 703: Calculate the distance between the starting geographic coordinates and the ending geographic coordinates to obtain the geographic distance corresponding to the initial image sequence.

[0122] According to the embodiments of the present invention, the starting geographic coordinates corresponding to the first first type of image captured by the line scan camera are obtained based on the initial image sequence, the preset image list and the preset address list, and the ending geographic coordinates corresponding to the last first type of image captured are obtained. Based on the difference between the starting geographic coordinates and the ending geographic coordinates, the geographic distance corresponding to the initial image sequence is obtained, and the true distance represented by the stitched image corresponding to the initial image sequence output by the line scan camera is determined.

[0123] In this embodiment of the invention, to better implement the image stitching method based on a line scan camera provided in this embodiment, an image stitching device based on a line scan camera is provided, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of an embodiment of the image stitching device based on a line scan camera provided in this invention. The image stitching device based on a line scan camera shown includes:

[0124] Address module 801 is used to obtain the starting geographic coordinates and ending geographic coordinates corresponding to the initial image sequence, and to determine the geographic distance corresponding to the initial image sequence. The initial image sequence includes multiple first-class images, where the first-class images are strip images.

[0125] The stitching module 802 is used to stitch together each first-type image in the initial image sequence to obtain the second-type image corresponding to the initial image sequence;

[0126] The adjustment module 803 is used to obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and the image length to obtain the target image corresponding to the initial image sequence.

[0127] In some embodiments of the present invention, the adjustment module 803 is further configured to obtain the image length of the second type of image; obtain the standard image length corresponding to the geographical distance; determine the adjustment ratio based on the difference between the image length and the standard image length; stretch or compress the second type of image according to the adjustment ratio, and use the stretched or compressed second type of image as the target image corresponding to the initial image sequence.

[0128] In some embodiments of the present invention, the stitching module 802 is further configured to determine a target heading angle based on the statistical characteristics of the heading angles of each first type of image, wherein the statistical characteristics include at least one of the median, mode, mean, and expected value of each heading angle; calculate the difference between the target heading angle and each heading angle to obtain the rotation angle corresponding to each first type of image; and stitch the first type of images together based on the rotation angle corresponding to each first type of image to obtain a second type of image.

[0129] In some embodiments of the present invention, the address module 801 includes:

[0130] The time unit is used to obtain the first acquisition time and the last acquisition time corresponding to the first type of image in the initial image sequence;

[0131] The geographic coordinate unit is used to obtain the starting geographic coordinates corresponding to the first collection time and the ending geographic coordinates corresponding to the last collection time, based on a preset address list.

[0132] The distance calculation unit is used to calculate the distance between the starting geographic coordinates and the ending geographic coordinates to obtain the geographic distance corresponding to the initial image sequence.

[0133] In some embodiments of the present invention, the time unit is further used to obtain the global encoding range corresponding to the initial image sequence, and to obtain the start encoding and end encoding corresponding to the global encoding range; to obtain the first acquisition time corresponding to the start encoding in the preset image list, and to obtain the last acquisition time corresponding to the end encoding in the preset image list.

[0134] In some embodiments of the present invention, the image stitching device based on a line scan camera further includes:

[0135] The acquisition module 804 is used to acquire the moving speed of the pavement inspection robot, acquire first-type images based on the moving speed, assign codes corresponding to the first-type images, and store the acquisition time of the first-type images and the codes corresponding to the acquisition time into a preset image list; if the number of first-type images in the preset image list reaches a preset threshold, then acquire each first-type image and the target code corresponding to each first-type image; associate each first-type image with the corresponding target code to obtain an initial image sequence.

[0136] In some embodiments of the present invention, the image stitching device based on a line scan camera further includes:

[0137] The segmented adjustment module 805 is used to obtain the global coding range corresponding to the initial image sequence; divide the global coding range into multiple local coding ranges according to the preset coding interval; obtain the local geographic distance corresponding to each local coding range; stitch together each first-type image within each local coding range to obtain a local second-type image; and obtain the local image length corresponding to each local second-type image; adjust each local second-type image according to the local geographic distance and the local image length to obtain the adjusted image corresponding to each local second-type image; and stitch together the adjusted images to obtain the target image.

[0138] According to the embodiments of the present invention, the geographic distance corresponding to the initial image sequence is determined based on the starting geographic coordinates and ending geographic coordinates in the initial image sequence. The second type of image obtained by stitching the first type of images in the initial image sequence is adjusted by the geographic distance so that images of the same resolution represent the same length information, thereby enabling the stitching of images of the same resolution.

[0139] This image stitching device based on a line scan camera may include components such as a processor 901 with one or more processing cores, a memory 902 with one or more computer-readable storage media, a power supply 903, and an input unit 904. Those skilled in the art will understand that... Figure 9 The image stitching device structure based on a line scan camera shown does not constitute a limitation on image stitching devices based on line scan cameras. It may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0140] The processor 901 is the control center of the line scan camera-based image stitching device. It connects various parts of the device via various interfaces and lines, and executes software programs and / or modules stored in the memory 902, as well as calling data stored in the memory 902, to perform various functions and process data, thereby providing overall monitoring of the device. Optionally, the processor 901 may include one or more processing cores; preferably, it may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 901.

[0141] The memory 902 can be used to store software programs and modules. The processor 901 executes various functional applications and data processing by running the software programs and modules stored in the memory 902. The memory 902 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the image stitching device based on the line scan camera. In addition, the memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 902 may also include a memory controller to provide the processor 901 with access to the memory 902.

[0142] The image stitching device based on a line scan camera also includes a power supply 903 that supplies power to the various components. Preferably, the power supply 903 can be logically connected to the processor 901 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 903 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0143] The image stitching device based on the line scan camera may also include an input unit 904, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0144] Although not shown, the image stitching device based on a line scan camera may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 901 in the image stitching device based on a line scan camera loads the executable files corresponding to the processes of one or more application programs into the memory 902 according to the following instructions, and the processor 901 runs the application programs stored in the memory 902 to realize various functions, as follows:

[0145] Obtain the starting and ending geographic coordinates corresponding to the initial image sequence, and determine the geographic distance corresponding to the initial image sequence. The initial image sequence includes multiple first-class images, where the first-class images are strip images.

[0146] By stitching together the first type of images in the initial image sequence, the second type of images corresponding to the initial image sequence are obtained;

[0147] Obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and image length to obtain the target image corresponding to the initial image sequence.

[0148] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0149] To this end, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the image stitching methods based on a line scan camera provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0150] Obtain the starting and ending geographic coordinates corresponding to the initial image sequence, and determine the geographic distance corresponding to the initial image sequence. The initial image sequence includes multiple first-class images, where the first-class images are strip images.

[0151] By stitching together the first type of images in the initial image sequence, the second type of images corresponding to the initial image sequence are obtained;

[0152] Obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and image length to obtain the target image corresponding to the initial image sequence.

[0153] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0154] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0155] Since the instructions stored in the storage medium can execute the steps in any of the image stitching methods based on a line scan camera provided in the embodiments of the present invention, the beneficial effects that any of the image stitching methods based on a line scan camera provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0156] The foregoing has provided a detailed description of an image stitching method, apparatus, device, and storage medium based on a line scan camera provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image stitching method based on a line scan camera, characterized in that, The method, applied to a pavement inspection robot, includes: Obtain the starting and ending geographic coordinates corresponding to the initial image sequence, determine the geographic distance corresponding to the initial image sequence, wherein the initial image sequence includes multiple first-type images, and wherein the first-type images are strip images; By stitching together the first type of images in the initial image sequence, a second type of image corresponding to the initial image sequence is obtained; Obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and the image length to obtain the target image corresponding to the initial image sequence; The step of stitching together each of the first type of images in the initial image sequence to obtain the second type of image corresponding to the initial image sequence includes: The target heading angle is determined based on the statistical characteristics of the heading angles of each of the first type of images. The statistical characteristics include at least one of the median, mode, mean, and expected value of each heading angle. Calculate the difference between the target heading angle and each heading angle to obtain the rotation angle corresponding to each of the first type of images; The first type of images are stitched together according to the rotation angle corresponding to each first type of image to obtain the second type of image.

2. The image stitching method based on a line scan camera as described in claim 1, characterized in that, The step of obtaining the image length of the second type of image, and adjusting the second type of image according to the geographical distance and the image length to obtain the target image corresponding to the initial image sequence includes: Obtain the image length of the second type of image; Obtain the standard image length corresponding to the geographical distance; The adjustment ratio is determined based on the difference between the image length and the standard image length. The second type of image is stretched or compressed according to the adjustment ratio, and the stretched or compressed second type of image is used as the target image corresponding to the initial image sequence.

3. The image stitching method based on a line scan camera as described in claim 1, characterized in that, The step of obtaining the starting and ending geographic coordinates corresponding to the initial image sequence and determining the geographic distance corresponding to the initial image sequence includes: Obtain the first and last acquisition times corresponding to the first type of image in the initial image sequence; Based on a preset address list, obtain the starting geographic coordinates corresponding to the first collection time and the ending geographic coordinates corresponding to the last collection time. Calculate the distance between the starting geographic coordinates and the ending geographic coordinates to obtain the geographic distance corresponding to the initial image sequence.

4. The image stitching method based on a line scan camera as described in claim 3, characterized in that, The process of obtaining the first acquisition time and the last acquisition time corresponding to the first type of image in the initial image sequence includes: Obtain the global encoding range corresponding to the initial image sequence, and obtain the start encoding and end encoding corresponding to the global encoding range; Obtain the first acquisition time corresponding to the start code in the preset image list, and obtain the last acquisition time corresponding to the end code in the preset image list.

5. The image stitching method based on a line scan camera as described in any one of claims 1 to 4, characterized in that, Before the steps of obtaining the starting and ending geographic coordinates corresponding to the initial image sequence and determining the geographic distance corresponding to the initial image sequence, the method includes: The movement speed of the pavement inspection robot is obtained, a first type of image is acquired based on the movement speed, a code corresponding to the first type of image is assigned, and the acquisition time of the first type of image and the code corresponding to the acquisition time are stored in a preset image list. If the number of images of the first type in the preset image list reaches a preset threshold, then each image of the first type is obtained, and the target code corresponding to each image of the first type is obtained; Each of the first type of images is associated with the corresponding target code to obtain an initial image sequence.

6. The image stitching method based on a line scan camera as described in claim 1, characterized in that, Before the steps of obtaining the starting and ending geographic coordinates corresponding to the initial image sequence and determining the geographic distance corresponding to the initial image sequence, the method includes: Obtain the global encoding range corresponding to the initial image sequence; The global coding range is divided into multiple local coding ranges according to a preset coding interval; Obtain the local geographic distance corresponding to each of the local coding ranges, stitch together each of the first type images within each of the local coding ranges to obtain a local second type image, and obtain the local image length corresponding to each of the local second type images; Based on the local geographic distance and the local image length, each of the local second-type images is adjusted to obtain the adjusted image corresponding to each of the local second-type images; The target image is obtained by stitching together the adjusted images.

7. An image stitching device based on a line scan camera, characterized in that, The device includes: The address module is used to obtain the starting and ending geographic coordinates corresponding to the initial image sequence and determine the geographic distance corresponding to the initial image sequence. The initial image sequence includes multiple first-type images, wherein the first-type images are strip images. The stitching module is used to stitch together each of the first type of images in the initial image sequence to obtain the second type of image corresponding to the initial image sequence; An adjustment module is used to obtain the image length of the second type of image, and adjust the second type of image according to the geographical distance and the image length to obtain the target image corresponding to the initial image sequence; The stitching module is further configured to determine a target heading angle based on the statistical characteristics of the heading angles of each of the first type of images, wherein the statistical characteristics include at least one of the median, mode, mean, and expected value of each heading angle; calculate the difference between the target heading angle and each of the heading angles to obtain the rotation angle corresponding to each of the first type of images; and stitch the first type of images together based on the rotation angle corresponding to each of the first type of images to obtain a second type of image.

8. An image stitching device based on a line scan camera, characterized in that, It includes a memory and a processor; the memory stores an application program, and the processor runs the application program within the memory to perform the operations in the image stitching method based on a line scan camera as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores multiple instructions, which are adapted for loading by a processor to execute the steps in the image stitching method based on a line scan camera as described in any one of claims 1 to 6.

Images